Petro Rabigh a Failure in Fundamentals Lead to a Cyber Attack
Abstract:
The Saudi Arabian, Petro Rabigh complex one of the largest petroleum and chemical production facilities in the world suffered a cyber-attack against the Safety Instrumented System (SIS); which is the industrial safety component of the Industrial Control System (ICS). This was a nation-state cyberattack against another nation’s critical infrastructure. The attack against the SIS was successful, in that malware was downloaded on to selected SIS controllers. However due to two incidents which prompted an analysis of the system that resulted in the discovery of the malware; the refinery and its production was not directly affected. This paper provides a brief discussion of the attack, response, and the after-action mitigation recommendations. A point in the discussion is that fundamental cybersecurity controls were not fully implemented specifically segmentation, and the rule of least privilege. In addition, the depth of the exploit and anti-forensic actions of the attacker revealed an extremely well-funded and highly competent advisory; one that given enough time would overcome most any barrier.
Keywords: Petro Rabigh, Triton, ICS, SIS
Table of Contents
The Petro Rabigh Cyber Attack Introduction 4
The Petro Rabigh Oil Refinery 5
The Attacker 5
The Attack 6
The Inject Steps 8
After Action 10
Conclusion 12
References: 13
Figures
Figure 1 Petro Rabigh, Sumitomo Chemical Ltd 5
Figure 2 ICS Network Overview, FireEye 6
Figure 3 Schneider Electric TRICONEX Switch 7
Figure 4. Schneider Malware Inject-1 9
Figure 5. Schneider Malware Inject-2 9
Figure 6. Schneider Malware Inject-3 9
Figure 7. Schneider Malware Inject-4 10
The Petro Rabigh Cyber Attack Introduction
In June of 2017 at the Saudi Arabian Petro Rabigh refinery complex, a system safety controller as part of the Safety Instrumented System (SIS) was tripped. The SIS is the companion instrumented control system to the refinery’s Industrial Control System (ICS). While the ICS is a network of sensors and controllers that are programed and designed to autonomously run a process that makes or controls something, the SIS is the companion system of controllers that maintain the safety of the ICS system. For the most part ICS systems automate processes. These processes range from the flow of drinking and irrigation water, to the production of oil, and gas in refinery. As the production process is automated, so too is the safety system that monitors the production effort.
The first indication that there was a problem with the facility was the anomaly of June 2017. The error reported by the SIS was traced to a malfunctioning Schneider Electric Triconex controller. The safety device shutdown or took offline a section of the facility. The Schneider Electronic technicians that were called in, but could not find a problem, they replaced the unit, and took the faulty device to their lab for analysis. The section of the facility that had been shutdown was restarted and brought back into production.
On August 4, 2017 again the Petro Rabigh refinery complex suffered another shutdown by the SIS. As with the June incident the controllers and operators on staff could not find any evidence of problems with the production system. Again, the Schneider Electronic technicians were recalled, and the results were non-conclusive. At this point Schneider Electronic and the management escalated the investigation and called in forensic consultants. With these additional resources the production ICS, SIS, and IT networks were investigated. With this review of the networks, their controllers, and the corresponding production systems, the investigation revealed evidence of a highly complex cyber-attack directed at the SIS system. At this point Schneider Electric reached out to industry and government which resulted in the discovery of a nation state attack against the refinery.
The Petro Rabigh Oil Refinery
The Petro Rabigh facility is located along the Red Sea, it is a major supplier of chemicals and petroleum products to Africa, Asia, and Europe. It is over 3,000 acres and is one of the largest oil and chemical production facilities in the world. It produces more than 5 million tons of petrochemicals a year, to include kerosene and gasoline. In addition, as an integrated chemical and refining complex it also produces other products to include antifreeze and common polypropylene plastics. The facility is a joint venture between Saudi Arabia’s Aramco and Japan’s Sumitomo Chemical Ltd.
The Attacker
The attack against the SIS system was from a source that was extremely well funded, considerably sophisticated and professional. The attacker had the resources to spend years in the exploit, it is estimated that the first penetration was in 2014 . In addition, the attacker had to have built a sandbox of Schneider Electric controllers, to include the Tricon Controller which was the target of the attack, and a principal controller of the SIS network. The TMR Tricon Model 3008 v10.3 controller, at this time had been in production for about sixteen years and is common in ICS systems throughout the world. In addition, the staff involved in the attack were most likely trained in how to program these controllers, and most likely had attended training provided by Schneider Electric. This would have included the Triconex operating system software. As a result, the attackers were able to deconstruct the Schneider TriStation protocol, rewrite the code and inject it into the controllers essentially without being compromised. The analysis by FireEye indicated that the Russians were behind the attack. As described, only a nation state could fund such a long term, three-year, effort and have the resources in terms of hardware and systems engineering.
The Attack
First it is important to state that the attack on the SIS system was successful as defined by being able to infect the controllers. However, it is also important to note that an attack against the facility was never executed. The intent of the campaign against the Petro Rabigh facility is based on assumption, and the actual path toward the malware injects is not absolute, but based on the available evidence, and assumptions.
Apparently, the attack began in 2014, after what is assumed much time in reconnaissance, and began with an exploit of a “poorly configured firewall ”. This firewall was on the IT network, which allowed the attacker to pivot to the DCS (Distributed Control System) or the ICS network, then to the SIS network. It is important to note that traditionally there has been a segmentation of these networks from one another. The main networks being the IT infrastructure, DCS or ICS networks, and the SIS networks. However, over time and for the sake of efficiency these networks had been crossed.
Once in the network the attackers directed their attention to the exploit. As mention, in the attackers sandbox they had developed and tested the Triton executable. This involved the development of an executable that would be implement by the Schneider TriStation protocol; and one that would be used to download the application malware code into the SIS controllers.
However, there were two Schneider barriers the attackers had to circumvent . First, was a physical switch that controlled the mode of the Triconex controller. Second, the code downloaded by the TriStation had to be persistent.
The Schneider Triconex will only accept a software download if the switch is in the program mode. However, the Schneider Electric Triconex operating system defines a “zero day” state, as a state in which privilege can be escalated; and with supervisor privilege the switch can be in any position. A technical fact that required a though understanding of the operating system. To take advantage of this, the malware code elevated itself from user to supervisor, and in doing so obtained read/write/execute privileges. In this state as the supervisor it was able to copy itself into memory.
In terms of persistency the code took advantage of the fact that industrial controllers are rarely if ever rebooted. Some controllers operate throughout their lifetime without ever being refreshed or rebooted. Consequently, being in RAM is as good as being in firmware. The malware was designed to find a free location on the controller, and then copy into the firmware memory. As mentioned though in RAM, the malware gained de-facto persistence due to the fact that SIS controllers are not rebooted, except by exception.
In order to establish itself in the controller the malware took advantage of another condition in the Triconex operating system. This allowed the malware to be called first and in doing so initiated the inject process for downloading the malware. In the supervisor role the malware registered itself to be called by the SIS operating system first. When it received a specific network command the malware payload would be called before the established Tricon communications protocol. In this fashion it was called first and established itself in memory without the physical switch.
The Inject Steps
The following slides are from the Schneider Electric debrief that was focused on the high-level steps the attacker took in taking control of the Tricon Controllers.
Once the attacker had gained access to the SIS controller network the objective was to download the malware. This process had been tested and executed on the attacker’s sandbox. At this point in the exploit the attacker was well aware of the SIS configuration, they had the same hardware and software in their facility. They had simulated the attack and had tested the software. In this phase the attacker is beginning to load their malware and validating the controller state.
The next step was to run exploit checks 1 and 2, and begin to write to the system memory
In this step the exploit begins to address the zero-day requirement, which is needed to escalate privilege to supervisor so that the malware can be written to memory. Here the payload is downloaded into memory, the establishment of the malware first in the network call, and system permissions are restored.
At this state the malware or the RAT (Remote Access Trojan) is loaded and is waiting for commands. This outlines the process when the controllers receive commands from the attacker to read/write/execute.
After Action
As discussed after the second incident Schneider Electric and the Petro Rabigh facility realized that this was beyond their capability to resolve and called industry and other governments for assistance. The incidents in June and August were most likely due to inadvertent errors. It appears that a bug in the malware tripped up the process and the SIS devices stopped and entered a failed state This disruption provided the opportunity for the stakeholders to evaluate and request assistance.
The following businesses and government agencies provided assistance:
Mandiant / Fire Eye Dragos FBI
Claroty Virsec DoD /DARPA
Securiosity McAfee DHS/INL
NCCIC
There are a number of after-action review recommendations, that are related to best practices in ICS networks. Nevertheless, as described earlier a dedicated adversary will be able to work through these defenses overtime. However, the counter is the concept of an active defense. Where, as the adversary maneuvers though the defenses, and tries to escalate privilege these overt activities are more likely than not to be identified. Consequently, in concert with defense in depth is the requirement for active monitoring. The after-action mitigations and recommendations are listed below, not in any order of importance:
Coordinated Action: Schneider Electric and the Petro Rabigh facility took action, and what would have been expected, did not ignore the fault discovered in August 2017. The success of uncovering this exploit was most importantly due to their mature leadership and transparency, and the international team of business and government working together.
Reinforcing Security: Schneider Electric has worked to secure the SIS infrastructure, it has examined the field installation of the Tricon 3008 controller, and is monitoring their behavior to determined if the same virus has infected other facilities, and it has made changes to the controller to reduce the vulnerabilities uncovered. In addition, the vulnerability is part of anti-virus applications.
Network Segmentation: As operational networks (IT, ICS, SIS) have grown, and as the requirement of real-time monitoring and control have grown, the segmentation of these networks have diminished. In this situation the attacker was able to exploit a misconfigured firewall on the IT network and move across domains to the SIS network.
Least Privilege: The principle of least privilege is the practice of limiting access rights to the bare minimum. This principal includes restricting the execution of unknown apps and tightly controlling the installation of new apps. The actions of strict privilege management, enhanced application whitelisting and controlled execution privileges can shrink the attack surface.
Configuration Management Control: The attack entry point was a misconfigured firewall; the concept of configuration management is to ensure all devices are properly configured.
Incident Response Review: Once the attack had been discovered there began the process of resetting user credentials. This included multifactor identification. However, the attacker had compromised the IT network and the system administration files. So that when the users tried to reconfigure their credentials through multifactor authentication, there cell phone numbers were compromised.
Conclusion
Perhaps in addition to the luck that is the failure of the malware prior to an attack, the second most important factor is that this incident was not hidden away. It was publicized and in the process of announcing this attack it demonstrated the importance of government and industry working together, across national boundaries to combat cybercrime. No one company could unravel this attack. As mentioned, the exploit and the anti-forensic actions of the attacker revealed a nation state actor who is an extremely well-funded and highly competent advisory.
The attack also illustrates the attack cycle and response, in terms of industry improving its defense. In addition, it pointed to the fundamentals as mentioned in the after-action review. Some of these fundamentals include the full implementation of principal of segmentation, the principal of the rule of least privilege, system monitoring, etc. The point being that ICS systems are not isolated or immune to highly sophisticated directed attacks, they too to have to constantly review and check their application of the principals of cyber defense.
References:
Retrieved from: https://hub.packtpub.com/fireeye-reports-infrastructure-crippling-triton-malware-linked-to-russian-government-tech-institute/ April 11, 2019
Retrieved from: https://www.eenews.net/stories/1060123327 March 7, 2019
Retrieved from: https://www.fireeye.com/blog/threat-research/2019/04/triton-actor-ttp-profile-custom-attack-tools-detections.html Apr 10.2019
Retrieved from: https://www.fireeye.com/blog/threat-research/2017/12/attackers-deploy-new-ics-attack-framework-triton.html Dec 14 2017
Retrieved from: https://www.midnightbluelabs.com/blog/2018/1/16/analyzing-the-triton-industrial-malware January 16. 2018
Retrieved from: https://www.virusremovalguidelines.com/tag/what-is-tristation-protocol June 8, 2018
Retrieved from: https://www.cyberark.com/threat-research-blog/anatomy-triton-malware-attack/ February 8, 2018
Retrieved from: YouTube video https://www.youtube.com/watch?v=f09E75bWvkk&feature=youtu.be January 21, 2018