How a SOC and CERT protect the digital city

Yesterday there was a major IT security incident at an organisation. When logging into the domain controller, often a crown jewel of the institution, someone realised that the actual administrator no longer had control of their account. Instead, an attacker had infiltrated and wreaked havoc on the network. This scenario can be compared to a fire in a building in a city that spreads intensively and spreads to the neighbouring buildings.

What sounds like a dramatic isolated incident happens in practice more often than you might think. In order to understand how we can prevent such a digital conflagration in an organisation or manage it as efficiently as possible, I would like to give you an understanding of this analogy:

Imagine an organisation's IT infrastructure as a digital city...

The situation

Today I'm sitting in the digital operations centre of Switch's Security Operations Centre (SOC). Equipped with virtual binoculars, a radio and - as always - a camomile tea, I'm ready to protect the digital cities from trouble.

As soon as I look through my binoculars, I realise that several buildings in the neighbouring city outside the walls of our city network are on fire. Unfortunately, this digital city is outside my sphere of influence, as it has decided not to join our operations control centre.

The emergency services

Our Computer Emergency Response Team (CERT) was alerted late in the evening by the city where the fire had broken out. This is how the process is defined: if an acute danger is recognised that cannot be dealt with alone, the emergency services are called - in the IT world the CERT - in the digital world the fire brigade.

There is still a blaze in our neighbouring town and a suspected arsonist is on the loose. The CERT is doing its best to recognise and extinguish the fires in the digital city and at the same time find out who started them and how, in order to prevent further fires. A difficult, complex task. Their goal: to contain the danger so that no more buildings start burning.

Under the circumstances described, however, this destination can only be reached via many detours. After 24 hours, it is still unclear exactly what has happened. The CERT has to painstakingly collate the data from the last few months. There is no operations control centre in the neighbouring city that has a picture of the situation in the city with its events and threats.

The measures to be taken by the organisation are not easy to determine, because without a precise picture of the situation, the consequences are difficult to assess, as if they were poking around in the fog.

Fires often remain undetected for a long time and the emergency services - as in our case - often arrive too late to extinguish the developing fire quickly and effectively. In addition, they have to identify the exact source of the fire in dense fog and smoke in order to direct the extinguishing water to where it is most urgently needed. The last of the firefighters are also worn down by the unclear situation. Could this have been prevented?

The operations control centre

The biggest problem in such situations is the lack of situational awareness. Without an overview, you don't know where there is smoke or where there is already a fire. You are faced with the following questions: Can threats be contained preventively? Do you need additional emergency services? Where should they be sent and what exactly should they do?

The boundaries between SOC and CERT often seem to be fluid, but especially in times of increasing IT security threats and growing IT infrastructures, their co-operation becomes crucial. Whereas in the past the CERT was used as the first line of defence in small villages and could keep track of the few houses, a SOC becomes essential as soon as the village has grown into a town with the necessary infrastructure for several districts.

Let's imagine the SOC as the situation centre of the digital city. Someone has to keep track of everything. This operations control centre has to monitor ongoing operations, potential threats and incoming information in order to deploy firefighters efficiently. It's a demanding job, with a flood of information pouring in that needs to be assessed. The trick is to identify the crucial signs in order to initiate the appropriate processes.

In order to exploit synergies and pool the right specialists, Switch developed a Community SOC as a digital operations centre for organisations from the education and research community. It serves as a digital situation centre. 

A SOC would have helped

On the technical side: the whole disaster could very probably have been prevented if the organisation had had a SOC. When an IT security incident occurs, a SOC not only provides the most important information and thus enables well-founded decisions to be made, but in the best-case scenario also discovers the incident much earlier so that a major fire cannot break out. How does it work?

• Among other things, the SOC monitors where network packets are sent from and to. Endpoint Detection and Response (EDR) agents on all systems and network data would have discovered the arsonist while roaming around.
• It checks who has gained access and where. The systems' security logs raise the alarm if someone tries to enter a building or system without authorisation - just like our digital arsonist.
• The SOC can see who is using which applications and when. If our troublemaker buys huge quantities of fuel in the neighbourhood, this would definitely be noticed and would have triggered an alarm.
• The audit logs also reveal who is trying to log in from where and when. If the attacker had tried to climb over the city wall, the sirens would have been wailing.
• DHCP logs show which device had which IP address at which time. It's like a digital residents' registration office for the city. If the bakery moves, this is updated immediately so that the fire brigade knows where to go if an alarm goes off.
• The DNS logs reveal which system wanted to connect to which domain. You can think of it like a digital address book. If a resident tries to correspond with a foreign city, the address look-up is noted. So if the arsonist had previously sent letters to a dubious organisation or person, this would have been noted in the DNS logs. It is as if our culprit had secretly written letters to dodgy acquaintances who were already under surveillance. The operations control centre would have noticed this immediately and raised the alarm.
• Firewall logs show which IP addresses from different networks have communicated with each other. If the arsonist had already scouted out suspicious behaviour beforehand, the alarm would have been triggered.
• Even if the attacker had tried to cover his tracks, the file logs would have reported who had opened, changed or deleted which files.
• And finally, vulnerability management provides insights into system weaknesses, similar to a building inspection that monitors dilapidated buildings and reports them to the operations control centre.

This collected information, analyses and alarms - the Security Incident and Events Management (SIEM) - form the technical core of a SOC. 

Just one step away from success

How would this whole case have played out if the organisation had had a SOC?

I imagine how I would have started my duty as "SOCie of the Day" days, maybe months, before. The SOCie of the previous day would have done the handover with me and told me that he had spotted a few small clouds of smoke and had extinguished them himself. There was evidence of an arsonist trying to climb the city wall and I should keep an eye on it. I sat down at my desk and looked at the monitor.

Indeed - the arsonist won't give up. I report him to all the city's security guards, who keep a close eye on the weak points in the city wall and make sure that he has no chance of getting over the wall. In order to repair the weak points, I inform the craftsmen and women so that they can further strengthen the city wall. In the meantime, I also inform the SOCs of the neighbouring towns so that they are also prepared.

Of course, our arsonist is not the only culprit. And so I spot the next attacker, better equipped with pickaxe and rope, on the other side of the city wall. But he won't succeed either - I'm ready! I make it even harder for him and he has no chance of even throwing a match over the wall. 

The SOC is not a CERT and the CERT is not a SOC

The SOC must not allow itself to be drawn into individual operations, otherwise it will lose sight of the overall situation. If you focus too much on one fire, you could overlook other sources of fire, which could then threaten the entire city. To summarise, the SOC protects the digital assets, maintains an overview and dispatches the CERT in an emergency to deal with major and acute threats. It is indispensable for making informed decisions and avoiding unnecessary fire drills. The latter not only wear down staff, but also cause considerable costs in various areas.

You see, I've barely had my camomile tea before I've already successfully fought off two arsonists. So it's fun to protect the digital city! And how do you protect your digital city?