Reliability, availability and serviceability

While RAS originated as a hardware-oriented term, systems thinking has extended the concept of reliability-availability-serviceability to systems in general, including software: • Reliability can be defined as the probability that a system will produce correct outputs up to some given time t. Reliability is enhanced by features that help to avoid, detect and repair hardware faults. A reliable system does not silently continue and deliver results that include uncorrected corrupted data. Instead, it detects and, if possible, corrects the corruption, for example: by retrying an operation for transient (soft) or intermittent errors, or else, for uncorrectable errors, isolating the fault and reporting it to higher-level recovery mechanisms (which may failover to redundant replacement hardware, etc.), or else by halting the affected program or the entire system and reporting the corruption. Reliability can be characterized in terms of mean time between failures (MTBF), with reliability = exp(−t/MTBF). ==Failure types==

Physical faults can be temporary or permanent: • Permanent faults lead to a continuing error and are typically due to some physical failure such as metal electromigration or dielectric breakdown. • Temporary faults include transient and intermittent faults. • Transient (a.k.a. soft) faults lead to independent one-time errors and are not due to permanent hardware faults: examples include alpha particles flipping a memory bit, electromagnetic noise, or power-supply fluctuations. • Intermittent faults occur due to a weak system component, e.g. circuit parameters degrading, leading to errors that are likely to recur. ==Failure responses==

Transient and intermittent faults can typically be handled by detection and correction by e.g., ECC codes or instruction replay (see below). Permanent faults will lead to uncorrectable errors which can be handled by replacement by duplicate hardware, e.g., processor sparing, or by the passing of the uncorrectable error to high level recovery mechanisms. A successfully corrected intermittent fault can also be reported to the operating system (OS) to provide information for predictive failure analysis. ==Hardware features==

Example hardware features for improving RAS include the following, listed by subsystem: • Processor: • Processor instruction error detection (e.g. residue checking of results) with instruction retry e.g. alternative processor recovery in IBM mainframes, or "Instruction replay technology" in Itanium systems. • Processors running in lock-step to perform master-checker or voting schemes. • Machine Check Architecture and ACPI Platform Error Interface to report errors to the OS. • Memory: • Parity or ECC (including single device correction) protection of memory components (cache and main memory); bad cache line disabling; memory scrubbing; memory sparing, memory mirroring; bad page offlining; redundant bit steering; redundant array of independent memory (RAIM). • I/O: • Cyclic redundancy check checksums for data transmission/retry and data storage, e.g. PCI Express (PCIe) Advanced Error Reporting (AER), redundant I/O paths. • Storage: • RAID configurations for hard disk drive and solid-state drive storage. • Journaling file systems for file repair after crashes. • Checksums on both data and metadata, and background scrubbing. • Self-Monitoring, Analysis, and Reporting Technology for hard disk drive and solid-state drive. • Power/cooling: • Duplicating components to avoid single points of failure, e.g., power-supplies. • Over-designing the system for the specified operating ranges of clock frequency, temperature, voltage, vibration. • Temperature sensors to throttle operating frequency when temperature goes out of specification. • Surge protector, uninterruptible power supply, auxiliary power. • System: • Hot swapping of components: CPUs, RAMs, hard disk drives and solid-state drives. • Predictive failure analysis to predict which intermittent correctable errors will lead eventually to hard non-correctable errors. • Partitioning/domaining of computer components to allow one large system to act as several smaller systems. • Virtual machines to decrease the severity of operating system software faults. • Redundant I/O domains or I/O partitions for providing virtual I/O to guest virtual machines. • Computer clustering capability with failover capability, for complete redundancy of hardware and software. • Dynamic software updating to avoid the need to reboot the system for a kernel software update, for example Ksplice under Linux. • Independent management processor for serviceability: remote monitoring, alerting and control. Fault-tolerant designs extended the idea by making RAS to be the defining feature of their computers for applications like stock market exchanges or air traffic control, where system crashes would be catastrophic. Fault-tolerant computers (e.g., see Tandem Computers and Stratus Technologies), which tend to have duplicate components running in lock-step for reliability, have become less popular, due to their high cost. High availability systems, using distributed computing techniques like computer clusters, are often used as cheaper alternatives. == See also ==