Appunti Computing Infrastructures

Notes computer infrastructure

Indice

• 0. Introduction to DataCenter 3
• 0.1 “To build or not To build” 3
• 1. Overview of computer infrastructures 5
• 2. DataWareHouse 6
• 2.1 Architecture 6
• 3. Server: datacenter as a computer 7
• 3.1 GPU, TPU and FPGAs 7
• 4.1 Disk abstraction and HDD 8
• 4.1 Hard drives 8
• 6. SSD - Solid State Disk 9
• 6.1 Garage collector 9
• 6.2 Mapping table size 10
• 6.3 Wear leveling 10
• 7. RAID disks 17.
• 7 Reliability calculation 13
• 8. Storage system 14
• 8.1 DAS 14
• 8.2 NAS 14
• 8.3 SAN 14
• 9. Dependability 15
• 9.1 Reliability block diagrams 16
• 9.2 Standby redundancy 17
• 10. Networking 18
• 11. Building infrastructures 19
• 11.1 Cooling system 19
• 12. Cloud computing 21
• 12.1 Server consolidation 21
• 12.2 Cloud computing 21
• 13. Virtualization 23
• 13.1 Implementation 24
• 13.1 Virtual machine managers (System VMs- Same ISA) 24
• 13.2 Virtualization technique 25
• 13.3 Containers 25
• 14. Performance modeling 26
• 14.1 Queueing network 26
• 14.2 Operational law 27
• 14.3 Performance bound 28

0. Introduction to DataCenter

A DataCenter is “a building or portion of a building whose primary purpose is to house a computer room and its support areas”

It is typically composed by:

• IT equipment: server\memories and network devices
• Facility equipment: energy supply, cooling system…
• Server Room: physical rooms

0.1 “To build or not To build”

Use your own datacenter or outsource (and in which modality) is one of the most difficult choices.

Enterprise datacenter is managed by the company providing the target services:

• Pro: asset of the company, compete control
• Cons: High initial capital investment, require know-how

if is not possible to achieve own datacenter we can still pursue local datacenter with colocation DataCenters, where large facilities are built to be used from multiple companies:

• Pro: still high control, low capital investment
• Cons: DataCenter is no more an asset and we can have some access restriction

Managed Hosting Platform is a datacenter managed by a third part. The customer rents IT equipment and infrastructure if needed:

• Pro: low capital investment, cost saving for small business, no need of know-how
• Cons: higher operational costs, lower control

1. Overview of computer infrastructures

“A Computing infrastructure is a technological infrastructure that provide hardware and software for computation to other system and services”

There are very heterogeneous computer infrastructure with their pro and cons, for a datacenter:

One kind of infrastructures could be fog and edge computer, that is associated with the location where the data is processed:

• In Edge computing the data is processed closest to the sensors
• In Fog computing the computing is moved to processors linked to a local area network

From an It perspective we define:

• Application Software: The application why this system exists
• Environment Software: Programs, libraries to control the physical resources and provide tools
• Physical Resources of the system

2. DataWareHouse

In last years computation and storage of data was moved to the cloud system, user experience improved with easier management system and multiple access points. From a vendors point of view Software as a service allows faster application development, deployment and update. Also some workloads are fitting better on a datacenter.

ServerSide computing led to the creation of a computing system: warehouse-scale computers (WSCs)

WSCs typically belong to a single organization, use a relatively homogeneous hardware and system software platform, and share a common system management layer. They run a smaller number of very large applications (cluster of services).

Traditional data centers instead host a large number of relatively small sized application of different players that tend not to communicate each other

With the passing of time also WSCs power public clouds computing system

Multiple DataCenters are often replicas of the same service to reduce user (geographically) latency and improve serving throughput.

For this point of view the world is divided into Geographic Areas (GAs) defined by geo-political boundaries. Customers see regions as the fines grain discretization of the infrastructure

Availability Zones (AZs) are finer grain locations within a single computing region (unit).

WSC workloads must be designed to gracefully tolerate large numbers of components faults with little impact on service level performance availability

2.1 Architecture

Hardware implementation of WSCs can be very different, but the architectural organization of these system is relatively stable and made of:

Servers are like ordinary pc with different form factors

Storage are disk and Flash SSDs, these devices are connected to the datacenter network and managed by complex distributed system

Network is exploited by dedicated equipment that allow interconnection among devices

Some important component of building and infrastructure like cooling system, power supply and failure recovery must be considered in design

3. Server: datacenter as a computer

Servers hosted in shelves that are the “basic building block” of WSCs. They are interconnected by hierarchical network and supported by cooling and power systems.

WSC uses a relatively homogeneous hardware and system software platform.

The form factor allows the server to fit into shelves: blade, rack and tower are the most common shelves.

Tower Server (like a traditional PC) is scalable and easy to upgrade, cheap and easy to cool. But it consumes a lot of space and provides low performance. Difficult cable management.

Rack Server, special shelves that allow their interconnection. Must conform to specific sizes. Pro: simple cable management, cost-efficient and failure containment. Cons: power usage and maintenance.

The rack is the shelf that holds the servers together. It is often convenient to connect the network cable at the top of the rack. Typically battery are not placed in rack (but power distribution units yes)

u=4.415 cm

Blade servers, “hybrid rack”, in which servers are placed inside blade enclosures. Typically blade servers are the most advanced ones, smaller but difficult to heat manage them and costly.

3.1 GPU, TPU and FPGAs

Specialized accelerators are needed nowadays. GPUs have lots of parallel simple computational units to guarantee a lot of throughput. The bottleneck of this architecture is the communication phase to sync them, so GPUs within the tray are connected using high-bandwidth interconnects such as NVlink.

GPUs are still relatively general purpose devices. TPUs are used for training and inference of ML. Each Tensor core has an array for matrix computations (MXU) and a connection to high bandwidth memory (HBM) to store parameters and intermediate values during computation

FPGAs are Arrays of logic gates that can be programmed (“configured”) in the field. They are Array of carefully designed and interconnected digital subcircuits that efficiently implement common functions offering very high levels of flexibility.

4.1 Disk abstraction and HDD

Disks can be seen by an OS as a collection of data blocks that can be read or written independently. Each Block is identified with a numerical address called LBA.

Typically the OS groups blocks into clusters to simplify the access to the disk that contains file and metadata. Metadata contains file info and the links to the LBA where the file content can be located.

To read\write : Access the metadata to locate its blocks, after Access the blocks to read\write its content

To delete a file: only update the meta-data

Since the file system can only access clusters, the real occupation of space on a disk for a file is always a multiple of the cluster size: a= ceil(s/c)*c where s = file size, c= cluster size, a = size of the disk file

if a file is too big to fit continuously in the memory, it is splitted into smaller chunks. The effect of splitting into a non-contiguous clusters is called external fragmentation

4.1 Hard drives

A hard disk drive (HDD) is data storage using rotating disks (platters) coated with magnetic material. Data is read in a random access-way.

Externally, hard divers expose a large number of sectors, individual sector writes are atomic. Sectors arranged into tracks. A cylinder is a particular track of multiple platters. A disk may have multiple, double-sided platters.

When we access data we have some delays:

• Rotation delay: time to rotate in the desired sector
• Seek Delay: time to move to a different track
• Transfer Time: time to read\write
• Controller overhead: management time

Formulas:

Full Rotation delay: = 1 / Average Time Rotation: = /2

Time seek = = /3 Service Time = = + + + / ℎ

Data Locality of data can improve the memory access time deleting the and time. Cache can improve memory access performance. Also, if there are a queue of requests to the disk, they can be reordered to improve performance:

• FCFS Scheduling: Serve request in order
• SSTF Scheduling: minimize seek time by always selecting the block with the shortest seek time. We can have the problem of starvation requests (solved considering a limited time window)
• Scan: move the head from inner to outer cylinder. Middle data are read more frequently.
• C-Scan: like scan, but only service requests in one direction (worst performance than SCAN)

Schedule can be implemented into: OS, On-Disk, DiskCommand Queue

6. SSD - Solid State Disk

Solid-state storage device, no mechanism or moving parts like HDD, built out of transistors.

We can consider a SSD as a matrix where we store bits. Into a single cell we can save 1 or more bits (single or Multi Level cell). The cost of the SSD decreases when we increase aggregation, however decrease speed and reliability.

NAND flash is organized into Paged and Blocks. A block contains multiple pages.

• Blocks is the smallest unit that can be erased
• Pages is the smallest unit that can be read\written. It has a state flag: Empty, Dirty, InUseSS