ITAM Handbook: Enterprise Infrastructure

This chapter of the ITAM Handbook explains the parts of enterprise infrastructure that matter for software licensing: server and CPU architecture, clustering, virtualisation, high availability, and management and monitoring, and closes with how these configurations affect licensing. The virtualisation section, with its pages on individual technologies, is on a separate page: Virtualisation and partitioning technologies.

Servers Architecture

Servers architecture has come a long way since the first mainframe systems of the 1950s. In the early days of computing, mainframe systems were large and expensive, requiring specialized rooms to house them and a team of trained professionals to operate and maintain them. As the cost of hardware decreased and the capabilities of microprocessors increased, server technology began to evolve rapidly.

Over time, servers have increased in the number of available processors and threads, enabling more complex workloads to be run on a single server. In the early days of computing, servers had a single processor, with a few exceptions, like the IBM mainframes, which had multiple processors. In the 1990s, Sun Microsystems released the UltraSPARC processor, which had multiple cores on a single chip. This allowed for more parallel processing and improved performance for specific workloads. Other companies, such as IBM and HP, also released multi-core processors during this time.

In the 2000s, Intel and AMD began producing processors with multiple cores, starting with dual-core processors and eventually moving to quad-core and beyond. Server manufacturers widely adopted these processors, becoming the norm for high-performance computing. Today, many server processors have dozens of cores and support hundreds of threads, allowing for extremely high levels of parallel processing.

The demands of enterprise workloads, such as big data processing and machine learning, have driven these improvements in processor technology. Companies like Google, Amazon, and Facebook have been leaders in this area, building massive data centers with thousands of servers running in parallel. These companies have also developed their own server hardware, software, and automation tools to manage the vast number of servers in their data centers.

Large data centers used by cloud providers have evolved to become highly automated and efficient. They are designed to host tens of thousands of servers and petabytes of storage, with built-in redundancy and fault-tolerance. The key to their success is the use of automation and orchestration technologies, which enable efficient provisioning, configuration, and management of the underlying infrastructure. Providers like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) use various technologies like Kubernetes, OpenStack, and Terraform to manage their data centers.

Organisations typically use commercial servers from companies like Dell, HP, and IBM for on-premise data centers. These servers are often configured in a cluster or blade configuration, with multiple servers working together to provide high availability and fault tolerance. Virtualization technologies are commonly used to maximise the use of server resources and reduce costs. Organisations may also use automation tools like Ansible or Puppet to manage and provision their servers.

Despite these differences, both cloud and on-premise data centers share a common goal: to provide reliable, secure, and scalable infrastructure to support the needs of the enterprise. In many cases, organisations will use a combination of both cloud and on-premise resources to meet their requirements. This hybrid approach allows them to take advantage of the benefits of cloud computing while still maintaining control over critical systems and data.

CPU Architecture

CPU sockets, processors, cores, threads, logical processors and hyperthreading

1. CPU Socket. A CPU socket is a connector on the motherboard that houses a processor. Each socket is designed to be compatible with a specific range or generation of processors. Sockets enable the physical connection and electrical communication between the processor and the motherboard.
2. Processor. A processor, also known as a Central Processing Unit (CPU), is the primary component of a computer responsible for executing instructions and performing calculations. It consists of multiple cores and can manage several tasks simultaneously.
3. Core. A core is an individual processing unit within a processor. Each core can independently execute instructions and perform calculations. Modern processors typically have multiple cores, allowing for improved performance and multitasking capabilities.
4. Thread. A thread is the smallest sequence of programmed instructions that can be managed independently by an operating system scheduler. Threads enable parallelism within a single process, allowing multiple tasks to be executed concurrently.
5. Logical Processor. A logical processor is an abstract representation of a processing unit that the operating system uses to schedule and manage tasks. Each core in a processor is seen as one logical processor by the operating system, but with hyperthreading enabled, each core is seen as two logical processors, allowing for more efficient task scheduling.
6. Hyperthreading. Hyperthreading is a technology developed by Intel that enables each core in a processor to execute two threads simultaneously. It creates an additional “virtual” core for each physical core, doubling the number of available logical processors and improving overall performance.

Instruction sets

The instruction set architecture (ISA) is the part of the CPU architecture that defines the machine language commands a CPU can execute. The two most common types of instruction sets are complex instruction set computing (CISC) and reduced instruction set computing (RISC). CISC processors have a larger and more complex instruction set, which allows them to execute complex operations in a single instruction. On the other hand, RISC processors have a smaller and simpler instruction set, making them faster and more power-efficient.

CISC architectures like x86 are commonly used in personal computers and servers. These architectures are designed to handle a wide range of workloads, including general-purpose computing, multimedia processing, and gaming. Some common partitioning technologies used in x86 servers include virtualisation and containerisation. Microsoft Windows and various Linux distributions are the most common operating systems used on x86 servers.

RISC architectures, such as SPARC, are commonly used in mobile devices, embedded systems, and other devices where power efficiency and low heat output are critical. These architectures are optimised for specific workloads like signal processing, multimedia, or networking. Some common partitioning technologies used in RISC-based servers include virtualisation and containerisation.

CPU architectures

Several different CPU architectures are available in the market, each with its own features and benefits.

• The x86 architecture is the most widely used CPU architecture and is found in most desktop and server systems. It is based on the CISC instruction set and is optimised for running general-purpose workloads, such as database applications, web servers, and virtualisation.
• The Power architecture is based on the RISC instruction set and is commonly found in high-performance computing systems and data centers. It is optimised for running data-intensive and compute-intensive workloads like financial modeling, scientific simulations, and artificial intelligence.
• The Itanium architecture, also known as IA-64, was designed by Intel and is optimized for running large-scale, mission-critical workloads such as enterprise resource planning (ERP), supply chain management, and customer relationship management (CRM). Itanium is based on the EPIC (explicitly parallel instruction computing) architecture, which emphasises parallelism and is highly scalable.
• The Solaris architecture, also known as SPARC, is a RISC-based CPU architecture that is commonly used in high-performance computing systems and large-scale data centers. It is optimised for running compute-intensive workloads like financial modeling, scientific simulations, and machine learning.
• The ARM architecture is a RISC-based architecture that is commonly used in mobile devices and embedded systems. It is optimised for low power consumption and is commonly found in smartphones, tablets, and other mobile devices.

Threads in different CPU architectures

1. Intel. Intel’s Hyperthreading technology allows each core to handle two threads simultaneously, improving resource utilisation and increasing parallelism and performance in various workloads. Itanium processors utilize Explicitly Parallel Instruction Computing (EPIC) to focus on parallelism at the instruction level, allowing for the simultaneous execution of multiple instructions within a single thread. They rely on compiler optimizations and hardware features to achieve parallelism but do not support simultaneous multithreading like other architectures.
2. AMD. AMD’s Ryzen and EPYC processors employ Simultaneous Multithreading (SMT), similar to Intel’s Hyperthreading, with each core executing two threads concurrently to enhance performance in multithreaded applications.
3. IBM. IBM’s Power processors support multiple threads per core using Simultaneous Multithreading (SMT). Support for 2, 4, or 8 threads per core depends on the processor generation, enabling efficient workload handling in various scenarios.
4. Oracle. Oracle’s SPARC processors use Simultaneous Multithreading (SMT) to enable multiple threads per core, with processors like the SPARC M8 handling up to 8 threads per core for increased throughput and efficiency in highly parallel workloads.

Clustering

Server clustering is a strategy employed in enterprise infrastructure to ensure high availability, load balancing, fault tolerance, and virtualisation by connecting multiple servers to work together as a single system. Clusters enable organisations to minimize downtime, optimize resource utilisation, and improve overall system performance. This section discusses the different types of clusters, their underlying technologies, and software licensing rules related to these configurations.

Types of clusters

There are several types of server clusters, each with a specific purpose and configuration:

• High Availability (HA) Clusters. High Availability clusters, also known as failover clusters, are designed to provide continuous operation with minimal downtime. They achieve this by detecting failures in the primary server and automatically transferring operations to a secondary server. HA clusters typically consist of two or more servers, with redundant components and shared storage.
• Load Balancing Clusters. Load balancing clusters distribute workloads evenly across multiple servers to optimise resource utilisation, minimise response times, and prevent system overloads. These clusters often employ a load balancer, which directs incoming requests to the appropriate server based on predefined rules and the current workload.
• Distributed Computing Clusters. Distributed computing clusters are designed to tackle complex computational tasks by dividing them into smaller tasks and distributing them across multiple servers. These clusters leverage the combined processing power of multiple servers to perform large-scale calculations and simulations efficiently.
• Storage Clusters. Storage clusters provide a centralised storage solution that consolidates data across multiple servers. They ensure data redundancy, fault tolerance, and high availability, while enabling seamless data access and management.
• Virtualisation Clusters. Virtualisation clusters enable the creation and management of virtual machines (VMs) across multiple physical servers. These clusters provide high availability and load balancing for VMs, allowing for efficient resource utilisation and simplified management of virtual environments.

Technologies used in clustering

Several technologies are employed in server clustering to facilitate communication, synchronisation, and management, including:

• Cluster Management Software: Software such as Microsoft Failover Clustering, Red Hat Cluster Suite, and Veritas Cluster Server help configure, monitor, and manage clusters.
• Load Balancers: Hardware or software load balancers, like F5 BIG-IP, HAProxy, or NGINX, distribute network traffic among servers in a load-balancing cluster.
• Virtualisation Platforms: Virtualisation platforms, like VMware vSphere, Microsoft Hyper-V, or Citrix XenServer, allow for creating and managing virtual machines across virtualization clusters.

Software licensing rules for cluster configurations

Software licensing rules may vary depending on the cluster type and the specific software vendor. It is essential to understand these rules to ensure compliance and avoid potential financial or legal issues.

• High Availability and Virtualisation Clusters: Software vendors, like Microsoft or Oracle, sometimes allow for license mobility within a high availability or virtualisation cluster. This means licenses can be transferred between servers within the cluster without additional costs. However, reviewing the specific vendor’s licensing terms is crucial to confirm the allowed mobility.
• Load Balancing Clusters: For load-balancing clusters, licensing rules may require licenses for each server in the cluster, regardless of the current workload distribution. Some vendors may offer specific licensing options for load-balancing scenarios, such as per-connection or per-request licensing.
• Distributed Computing Clusters: In distributed computing clusters, licensing may be based on the total number of cores or processors used within the cluster. It is essential to account for the combined processing power of all servers when determining the required licenses.

Virtualisation

The handbook’s virtualisation section explains the difference between hardware partitioning and software virtualisation, and describes HP-UX nPar/vPar, IBM LPAR, Microsoft Hyper-V, Nutanix, Oracle Solaris LDOMs and Zones, Oracle VM and VMware. It is kept as its own page: Virtualisation and partitioning technologies.

High Availability

High availability (HA) is a critical aspect of disaster recovery planning in enterprise IT. It refers to the ability of an IT system or application to continue functioning with minimal disruption, even when one or more components fail or become unavailable.

To achieve high availability, enterprise IT infrastructure is typically designed with redundant components and failover mechanisms. For example, a system might have redundant power supplies or network interfaces so that the other can take over seamlessly if one fails. Similarly, a cluster of servers might be configured so that if one fails, the others can pick up their workload without any downtime or data loss.

One common approach to achieving high availability is using load balancers. Load balancers distribute incoming network traffic across multiple servers or applications, ensuring that if one fails or becomes overloaded, the others can take on its workload. Load balancers can also be configured to monitor the health of servers or applications and redirect traffic away from those experiencing problems automatically.

Another important aspect of high availability is data replication. By replicating data across multiple locations or even across multiple data centers, enterprises can ensure that critical data remains available even in the event of a disaster. Replication can be done in several ways, such as synchronous or asynchronous replication. It can be combined with other techniques, such as data compression and encryption, to optimise performance and security.

Virtualisation technologies also play an essential role in high availability. By virtualizing servers, applications, and data, IT teams can create highly resilient environments that can be easily moved and replicated as needed. For example, virtual machines can be replicated across multiple hosts, allowing them to continue running even if one fails. Virtualization also enables the creation of disaster recovery sites, where critical workloads can be moved in the event of a significant outage.

Overall, high availability is essential to disaster recovery planning in enterprise IT. By designing systems and applications with redundancy, failover mechanisms, load balancing, and data replication, enterprises can minimise downtime and ensure critical services remain available despite unexpected events.

There are several enterprise software products used to ensure the high availability of business systems, including:

• Oracle Data Guard. This is a high-availability solution for Oracle databases that provides disaster recovery and automatic failover capabilities.
• Microsoft SQL Server Always On. This is a high availability and disaster recovery solution for SQL Server that provides automatic failover, replication, and backups.
• IBM PowerHA. PowerHA is a high-availability solution for IBM Power Systems that provides continuous availability for critical applications and services.
• VMware vSphere High Availability. This is a feature of VMware vSphere that provides automated recovery for virtual machines in the event of a server failure.
• SAP HANA System Replication. This is a high availability and disaster recovery solution for SAP HANA that provides automated failover and replication between systems.
• Red Hat High Availability. This is a suite of high availability technologies for Red Hat Enterprise Linux that provides clustering, failover, and load balancing capabilities.

These products are designed to ensure that critical business systems are always available and can recover quickly in the event of a failure or disaster. They provide features like automatic failover, replication, and load balancing to minimise downtime and ensure the continuity of operations.

Management and Monitoring

There are a variety of critical data points that are collected and monitored for infrastructure management. Some of the most important ones include the following:

• Resource Utilisation. This includes CPU, memory, disk, and network utilisation. Monitoring these metrics can help ensure that resources are being used efficiently and that there are no bottlenecks or performance issues.
• Availability. Monitoring the availability of servers, applications, and services is critical for ensuring uptime and minimising downtime.
• Security. Monitoring security-related data points such as failed login attempts, suspicious network traffic, and other anomalies can help detect and prevent security breaches.
• Capacity Planning. Capacity planning involves monitoring resource utilisation trends over time and predicting future resource needs. This helps ensure that no resource shortages could lead to performance issues or downtime.
• Event Logs. Monitoring event logs can help identify and troubleshoot issues that may occur in the infrastructure. Event logs can provide insights into system crashes, application errors, security breaches, and other issues.
• Compliance. Many organisations are subject to compliance regulations, such as HIPAA, PCI-DSS, and GDPR. Monitoring compliance-related data points can help ensure the organization meets regulatory requirements and avoids potential penalties.
• Performance. Performance monitoring involves collecting data on application response times, database query times, and other performance-related metrics. This helps ensure that applications and services perform optimally and that no performance issues impact end users.

Large enterprises use a variety of products for the management and monitoring of their infrastructures. These products allow organisations to ensure that their systems are running efficiently and reliably and to identify and resolve any issues that arise quickly. Some of the most commonly used management and monitoring products in large enterprises include the following (an illustrative list of product families as named in the handbook, not an endorsement):

These enterprise-grade management and monitoring solutions provide a comprehensive view of IT infrastructure, applications, and services, enabling organisations to ensure high availability, reliability, and performance. They offer advanced automation and analytics capabilities, allowing organisations to quickly identify and resolve issues before they impact the business. Additionally, they provide centralised management and reporting capabilities, enabling IT teams to manage and monitor IT infrastructure across on-premise, cloud, and hybrid environments.

IT Infrastructure and Licensing

This section explores how various enterprise infrastructure configurations, such as CPU architecture, server architecture, clustering, and virtualisation, can affect the licensing of enterprise software from vendors like Oracle, IBM, Microsoft, VMware, and SAP. Understanding the impact of these configurations on licensing is crucial for organisations to ensure compliance and optimise software costs.

CPU architecture

CPU architecture can impact software licensing as some vendors have specific licensing terms based on processor type. For example, IBM uses Processor Value Units (PVUs) for some software licensing, which assigns a value to each core based on the processor type and performance capacity. Oracle also differentiates between various processor types, like x86 and SPARC, and has different licensing multipliers for each.

Server architecture

Different server architectures, including physical, virtual, and cloud-based servers, can affect licensing. For instance, Microsoft offers various license mobility options for cloud-based deployments, allowing customers to transfer their existing licenses to cloud environments like Azure. SAP, on the other hand, requires customers to purchase specific cloud licenses for their cloud-based solutions like SAP HANA Enterprise Cloud.

Clustering

Clustering can influence software licensing, as some vendors may require additional licenses for clustered environments. For example, Microsoft SQL Server requires customers to license all nodes in a failover cluster, even if some nodes are passive. Oracle RAC (Real Application Clusters) also requires separate license and database licenses to enable clustering capabilities.

Virtualisation

Virtualisation can significantly impact software licensing, as vendors have different policies for virtualised environments. For example:

• Oracle allows soft partitioning of their products in virtual environments, but the entire physical host must be licensed, regardless of the number of VMs running Oracle software.
• VMware vSphere requires a separate license for each physical processor on the host, but customers can run unlimited VMs on that host without additional licenses.
• Microsoft Windows Server uses a core-based licensing model where customers must license all physical cores on the host, with a minimum of 16 cores per server. However, depending on the edition, a certain number of VMs may be covered by the license.

In conclusion, understanding how enterprise infrastructure configurations like CPU architecture, server architecture, clustering, and virtualisation impact software licensing is crucial for organisations to ensure compliance and optimize software costs. Different vendors have distinct licensing terms and requirements, so it is essential to carefully review and manage these aspects in the context of the organization’s infrastructure.