Real-Time Operations

Real-Time Operations


Thriving in dedicated environments, embedded real-time operating systems (RTOS) are growing more connected and omnipresent as they add functionality and the highest levels of security to their range.

From a trusted foundation controlling avionics, aerospace, industrial process control and telecommunications, embedded RTOSs are expanding their reach with the highest levels of security where there is a need to reduce costs by using virtualization.

RTOS are the “real deal” when it comes to real time. “Hard” real-time, safety-critical systems have the least latency possible, serving application requests and switching tasks in a matter of microseconds. If the need for speed is not as great, milliseconds in soft real time will suffice. To achieve FAA certification for safetycritical flight control systems, hard real-time operating systems must be deterministic, or consistently predictable.

“Real-time operating systems support building real-time hardware and software with microsecond, not millisecond, deterministic response time. It could be fatal if they don’t handle it, so it has to be 100 percent guaranteed,” said David Kleidermacher, chief technology officer of Green Hills Software.

Embedded RTOS play a vital role on the battlefield, providing encrypted information in hard real time to joint warfighters. The Joint Tactical Radio System (JTRS) uses the Integrity RTOS from Green Hills Software as the engine behind many tactical radios at the edge of the network, managed by the Network Enterprise Domain (NED).

Green Hills’ products and services are the basis for the secure, reliable interoperation between networking and legacy waveforms as well as critical radio network management and services. The devices can route and retransmit services between networking and legacy waveforms without interruption or denial of service.

Beyond the battlefield, embedded RTOS manage safety-critical operations from flight control systems to nuclear power plants and traffic lights. “An RTOS has to be simultaneous, like anti-lock brakes,” noted John Blevins, director, product marketing, LynuxWorks.

To meet real-time requirements, embedded RTOS guarantee consistent performance serving real-time application requests. As such, RTOS contain sophisticated scheduling algorithms for preemptive priority. Smaller than general purpose operating systems, RTOS are compact and highly efficient.

Two crucial design qualities in safe and secure RTOS are the separation kernel and the hypervisor. The separation kernel, or a similar design called the partitioning kernel, allocates resources into high assurance partitions with information flow controls that cannot be bypassed, meaning no component can use another’s communication path. This limits the damage that could be caused by viruses or bugs. Built-in security monitors check each access point individually.

The hypervisor provides multiple virtual machines supporting multiple operating systems on a single processor. However, the operational reality is usually two operating systems. “The technology supports a three-domain system, but it’s usually two domains in the military today,” noted Robert Day, LynuxWorks vice president of marketing.

Most real-time operating systems are embedded, but not all embedded systems are real time. As their name suggests, embedded systems are implanted in hardware.

Levels of Security

A case in point is the Navy’s Common Display System (CDS), part of the service’s Open Architecture Computing Environment. The Navy CDS uses the LynuxWorks LynxSecure separation kernel and hypervisor in its display console to provide an environment in which multiple operating systems running at different security levels—from Top Secret to unclassified— execute concurrently.

“The Navy CDS is a multi-domain, ship-based console that is tactically deployed and requires multiple, independent levels of security,” said Robert Day, LynuxWorks vice president of marketing.

“Safety-critical operating systems in the avionics world deal with fault conditions in FAA certification. They are designed to look at errors and faults, but they have been physically separate and not connected. Now that more connections like WiFi are being added, one has to worry about not only safety but security. The convergence of safety and security hits our sweet spot,” he said.

The CDS is a configurable, high assurance workstation that provides users with access to multiple shipboard applications at once. LynxSecure supports 64-bit addressing for high-end scalability and is now undergoing National Security Agency certification evaluation.

“A military customer is currently conducting a system level certification and accreditation for SABI/TSABI, as the NSA shifts its EAL [Evaluation Assurance Level] program,” said Blevins. TSABI is Top Secret and Below Interoperability while SABI is Secret and Below Interoperability.

LynuxWorks technology is POSIX-compliant. The Portable Operating System Interface (POSIX) is a set of standards specified by the Institute of Electrical and Electronics Engineers for compatibility between operating systems.

Another LynuxWorks embedded-RTOS product that complies with the POSIX standard, the LynxOS-178 family, received FAA approval for DO-178b Level A reusable software components. This allows developers to reuse the software across multiple safety-critical systems without the need for recertification.

LynxSecure conforms to the Multiple Independent Levels of Security/ Safety (MILS) architecture, adhering to the data isolation, damage limitation and information flow policies identified in MILS. Most of the high security functions are performed by the separation kernel. The partitions and information flow policies are defined by the kernel’s configuration. “The total source code base of LynxSecure is only 128KB, a tiny dynamic memory footprint,” noted Blevins.

To build fast, small RTOS software code, the JTRS program uses Green Hills MULTI Integrated Development Environment (IDE) tools. The MULTI IDE provides third party integration and the ability to generate fast, small code for 32- and 64-bit processors. The program also uses Green Hills’ GateD family of routing, switching and network management solutions. In addition, there is a debugger for multi-core systems to prevent the most common causes of software bugs. The tools are processor and platform-neutral.

The Green Hills technology used by the JTRS has achieved POSIX-conformance and much of it is NSA Type 1 certified for military grade cryptography, used to secure classified information. Type 1 certification is not a published standard. Type 2 products endorsed by the NSA deal only with unclassified information.

“As you build more sophistication into virtualization and RTOS, NSA Type 1 certified cryptography comes into play. Green Hills is a subcontractor to General Dynamics, Rockwell Collins and other major defense contractors. JTRS is also a direct customer of Green Hills,” noted Kleidermacher.

Secure Linux

The JTRS program is developing an open architecture to allow multiple radio types—including handheld, ground-mobile, airborne and maritime—to communicate with each other and link warfighters to the Global Information Grid. The goal is to produce a family of interoperable, modular, software-defined radios that operate as nodes in a network for secure wireless communication. JTRS includes integrated encryption and wideband networking software to create mobile ad hoc networks.

JTRS developments illustrate the growth of the Linux operating system in embedded systems. “JTRS wanted to deploy secure Linux in radio. Green Hills won that with the Integrity 178B operating system for flight safety,” said Kleidermacher.

“The Integrity 178B operating system is the highest safety- and security-certified commercial operating system today, as it has been certified by the NSA-managed NIAP lab to EAL6+/High Robustness—the highest Common Criteria security level ever achieved for software—and the FAA’s highest certification for safety-critical avionics, DO 178b, Level A. No other operating system has achieved both of the highest-level safety and security certifications,” said Kleidermacher.

Lockheed Martin is using Green Hill’s Integrity 178B RTOS and AdaMulti IDE (Ada programming language) to develop safety and security-critical software for the F-35 Joint Strike Fighter. The Integrity 178B is running in multiple airborne, Power Architecture-based systems.

As if to highlight the persistent growth of embedded RTOS, in 2009 Intel Corp. acquired Wind River, a leading embedded RTOS provider. Intel’s publicly stated intention was to grow its processor and software presence beyond the PC and server market into embedded systems and mobile handheld devices. Wind River retained the right to operate with processor-neutrality under the purview of Intel’s Software and Services Group.

Wind River’s flagship embedded RTOS, VxWorks, is a key technology for the X47B in the Navy’s Unmanned Combat Air System Carrier Demonstration (UCAS-D) program. Northrop Grumman chose VxWorks as the safety-critical software platform for the UCAS-D while GE Aviation chose VxWorks for the Common Core System, the backbone of the UCAS-D computers, networks and interfacing electronics. The X-47B demonstrated that an unmanned, tailless aircraft could operate refuel in flight.

“All systems of this nature, not just the UCAS, assure that the highest priority task or thread that is ready to run in the queue will run. They are designed for interrupt-driven systems responding in deterministic real time. With this foundation, you can build complex systems on top of it,” said Chip Downing, senior director, Wind River aerospace and defense.

Interrupt Driven

Interrupt-driven systems are used for scheduling efficient multitasking in real time operating systems. There are both hardware and software interrupt-driven systems, interrupt requests, handlers and masking, to assure the highest degree of reliability and predictability. While interrupts are masked, the current task has exclusive control of the CPU and is protected from any other task or interrupt from taking control.

VxWorks MILS is MILS-certified and under evaluation by NSA now for EAL 6+ certification, Downing said. Wind River’s Linux Secure is Common Criteria EAL 4+ certified and FIPS 140-2 certified for secure cryptography.

Intel has given Wind River the ability to expand the reach of its technology. “The Intel acquisition allowed us to go into the lowerpower embedded chip market in handheld devices. We are now putting our RTOS on more Intel chips. And as chips get more powerful, they run not just an application but an application environment—like an operating system,” said Downing.

As embedded developers use virtualization to run multiple operating systems, to simplify the porting of legacy applications onto new platforms, they can get a little help from Intel’s Virtualization Technology (VT). As part of Intel’s vPro technology, Intel VT provides hardware-assist by performing some virtualization tasks in hardware to reduce the overhead and improve the performance of virtualization software. For example, switching between two operating systems is faster when memory address translation is performed in hardware rather than software.

LynuxWorks is eyeing the military mobile and IT enterprise markets to expand its embedded RTOS reach. “We see the equivalent of Intel VT now going into mobile devices, such as an Advanced RISC Machine (ARM) processor with virtualization in the next generation of chips. Since smartphones, including Android phones, are already 90 percent ARM, the addition of virtualization in ARM processors will make mobile devices look more like laptops. So what remains is a matter of security. And there is a huge potential market for secure, mobile Department of Defense applications,” said Day.

“The warfighters have to carry about 10 cell phones with them now, but DoD wants to equip them with one device with multiple levels of security. The problem is that the architecture for cell phones is not designed for multiple security domains, so the market has to get to the point where mobile devices have the same hardware and virtualization technology that will enable multi-domain security,” Day said.

ARM is a 32-bit reduced instruction set computer (RISC) developed by ARM Holdings. It is now the most widely used processor in mobile phones and other common embedded processors.

As if to gear up for the broadening embedded RTOS mobile market, Wind River recently exercised its chip-neutral diversity, adding support for a new ARM system-on-chip (SoC) on its VxWorks RTOS and Wind River Workbench development tools.

The Xilinix Zynq-7000 Extensible Processing Platform combines an ARM Cortex-A9 processor based SoC with a field programmable gate array (FPGA), which is designed to be configured by the customer after manufacturing. Wind River is also working with Xilinx on Linux efforts “In the FPGA fabric, we can put extra things in the hardware in a customized chip to support Android on one core and a military communications channel in another core running VxWorks, for a nonsecure side and a secure side respectively,” said Downing.

LynuxWorks is moving toward the IT enterprise with its longproven safe, and now highly secure, RTOS. “Our software now runs on Intel dual core and quad core i3, i5 and i7 chips with Intel VTX virtualization support,” said Blevins. “We can move to enterprise IT since it’s the same hardware.”

However, much of the installed base in the IT enterprise does not have the same real-time, high security requirements for virtualization as safety-critical embedded RTOS. “It really is in the eye of the beholder where the line between hard and soft real time is drawn. It depends on the application requirements,” said Stephen Balacco, director, embedded software and tools practice, VDC Research Group.

Cloud Security

Despite its popularity, cloud computing has received criticism for a lack of security in its virtualization. As a result, VMware, a leading DoD IT enterprise cloud provider, has boosted security in its EXSi virtualization technology with a family of products called vShield.

“VMware has made great strides in the security space in the last few years with virtualization-aware security products like the vShield Edge, a virtualized firewall, and vShield App, which protects applications in the virtual data center against network-based threats,” said Rob Randell, principal security and compliance solutions architect at VMware. “VShield Endpoint provides file system protections, such as antivirus, file integrity monitoring, application whitelisting and data loss prevention.”

“The enterprise data center does not generally have an embedded or real time requirement. It has historically been driven by a need to reduce costs and consolidate multiple systems onto a single piece of hardware using virtualization technology. However, due to an increase in cyber-crime, network connectivity and multi-tenancy cloud computing, we see the military IT enterprise market evolving towards a requirement for very strict security,” said Blevins.

Security remains an obstacle to the growth of the military mobile RTOS market. However, early this year a much-needed boost arrived for Android security when NSA released Security Enhanced (SE) Android, which provides stricter access control policies.

Since Android is based on Linux, it made sense when the NSA ported its SE Linux to Android. However, to build SE Android, developers need to download the Open Source Project source code and sync it up.

“Fundamentally, you can’t retrofit a high level of security to Android or any other operating system that wasn’t designed for it. But you can retrofit at the system level by inserting software of trust underneath Android. We develop Android in a virtual machine partitioned in its own area. This could be used by a military service asking for a dual persona handheld device with two Androids. One would be used for sensitive information and situational awareness and other for the soldier’s personal quality of life,” said Kleidermacher.

“Integrity creates memory and time partitions with memory areas exclusively owned by each application and guaranteed resource availability. In addition, the encryption component always gets what it needs so there is no risk of leaking information. However, not every RTOS does partitioning. The microprocessor must have an MMU to do partitioning, and we have found that the military is most interested in the MMU, he said. “We are also looking at how to use off-the-shelf mobile devices that we tailor to military missions.”

MMU is a memory management unit, a hardware component that is responsible for managing access to memory requested by the CPU.

Clearly, there is much work underway to render Android militarygrade secure. “There needs to be expert testing of the Android security implementation on the target device. Of course, some testing can be done manually. However, using industry-leading automated test tools such as Wind River Framework for Automated Software Test for Android can deliver significant gains in test efficiency,” said Chris Buerger, senior director, Wind River¬†general purpose systems.

As embedded, secure RTOS-based mobile devices—such as the Green Hills’ JTRS tactical radios—populate the edge of the cloud, they will continue to pull the cloud out until they have created an embedded cloud. That is already a term in use for a highly reliable, lightweight computing structure with web services and applications dedicated to serving embedded RTOS.

“We’re starting to see embedded RTOS mobile devices connected to the cloud on the edge of networks. Because of the types of devices— smartphones, controllers and sensors with small compact RTOS and iPads—we see them going through the cloud back to enterprise systems. And that will take the forefront,” said Balacco.

Real-Time Sailing

Surface ship and submarine combat systems count on real-time data distribution to assure timely target accuracy. These systems and communications interfaces are continuously upgraded and refined to keep pace with hardware improvements. To integrate technology upgrades rapidly into legacy systems, Real Time Innovations (RTI) deploys its highly flexible, standards-based software called the Connext product family.

“RTI Connext is currently deployed in most naval surface combat systems. We are in the SSDS, Aegis and LCS combat systems as well as the LPD ship system network,” said Gordon Hunt, RTI chief applications engineer.

The SSDS is Ship Self-Defense Systems, a Raytheon combat management system deployed on carriers and other amphibious flat tops. Aegis, now a Lockheed Martin program, is the combat system on destroyers and cruisers. The LCS is the littoral combat ship and the LPD is the landing platform dock.

At the heart of the RTI Connext family is the company’s distributed networking, standards-based DataBus connecting data across systems, networks and devices, whether on embedded real-time platforms or enterprise servers.

“These are infrastructure systems of scale that understand the context and the expected behavior of data. With legacy systems, the management of data is built into the application. When you bring new capability to the table, our infrastructure makes sense of, describes and manages data behaviors as part of the infrastructure on the bus and it is all standards-based,” said Hunt. “The binary protocols are rigorously defined as open but are as efficient as proprietary binary protocols.”

Connext DDS is a distributed real-time bus with an application programming interface that complies with the Object Management Group’s Data Distribution Service (DDS) specification. The high performance product also provides quality of service support for both real-time and enterprise systems.

Connext Integrator is a flexible infrastructure for building integration with real-time performance across diverse protocols and legacy applications. “These are peer-to-peer systems with no server or central hub. We leverage every bit of hardware capability we have. Before we send the data, we are aware of what is important to the receiver. It’s about understanding data and its behavior relevant to an application’s use,” said Hunt.

The Integrator provides support for various standards, including Java Messaging Service, SQL databases and others. The database service integration includes Oracle, MySQL and other relational databases. There are also tools for visualizing, debugging and managing systems in real time, protocol conversion and an adapter software development kit.

In addition, Connext Integrator offers data transformation, content-based routing as well as bridging between local and wide area networks, unsecured and secured networks, and IPv4 and IPv6. The Integrator provides bidirectional integration between a relational database, Connext DDS or another RTI product called Connext Messaging, which is messaging middleware with tools and scalability extensions for developing applications that leverage embedded and enterprise design. ♦

Back to Top