Incident Response @ Big Four
┌──(insider@thread)-[~]
└─$ cat summary.txt
6 years experience
Incident response [IR] and security operations as a consultant.
When you’re doing forensics you are dealing in probabilities. There is the classic problem of you can’t ever be certain. What do you do?
Incident response and security operations as a consultant. Incident response is anything from planning, preparation and response to incidents. Security operations includes SOC [Security Operations Centre] transformation, operationalising the SOC and making it more efficient.
How do those relate to each other?
They are all on the defensive side. You need to have these teams communicate and collaborate constantly. If you don’t have one it would be hard to do the other.
How did you get into IR?
It was just by chance. I started as part of a detection engineering and threat hunting team that got merged into incident response.
It was probably the best way to get involved into incident response. Starting with threat hunting taught me to spot suspicious activity in a safer environment. But during incidents when the worst has already happened or is happening, the threshold for mistakes is lower.
What is a misconception about IR ?
Everyone thinks it’s this interesting James Bond style thing. It’s a lot less exciting, a lot of excel. You have huge amounts of data and you need to use it to reconstruct what an attacker did.
What do you find most rewarding?
The impact. You are responding to attacks that impact peoples’ day to day lives. I like the crisis and incident side of things.
There are incidents that can be categorised as breaches, where the attacker didn’t action on their objectives. For incidents that materialise and result in crisis scenarios, the business operations are impacted.
That means the business is no longer able to provide what they are providing, or public sector, they can’t deliver what they are supposed to deliver to citizens. Going to your doctor is something everyone can relate to. The doctor needs to use a system to give you medication, write your prescription and get your history up so they can properly investigate. Imagine if those systems are down.
How do you manage a crisis scenario?
That’s probably the most stressful situation for a client out there. There is huge pressure on you. It’s about constant communication, being honest, providing real updates and managing the expectations.
Everything is burning, everyone is calling you, they are trying to find out what’s happening. In a crisis situation you normally talk to the client three or four times a day in short updates just to discuss about progress, next steps, where we are, and what we found out. But also getting the download from the client on what they have been working on the latest developments.
What are some challenges of IR?
When you’re doing forensics you are dealing in probabilities. There is the classic problem of you can’t ever be certain really.
You need to be careful around probabilistic language. One that I learnt over time (painfully) is never use “there is no evidence of”. It’s always “We haven’t identified evidence of” because you can’t be certain there is no evidence, maybe you just missed it.
You said in forensics you are dealing in probabilities, what do you mean?
You can never have the confidence that your data is capturing everything that an attacker is doing. There are things that are not captured depending on the data you are relying on. So sometimes you need to make informed guesses or come up with hypotheses.
EDR / XDR [Endpoint Detection and Response / Extended Detection and Response] is designed to record as much data as possible that you will use during incidents or for detection. But if we’re analysing classic Windows artefacts, that data hasn’t really been designed to be a forensics log or a security log. It’s used by Windows for different functionality and we are just lucky enough to be able to use it for reconstructing what is happening on a system.
Can you give an example of what you mean by artefact reconstruction?
Probably the easiest example is prefetch. It’s included in Windows to preload the execution of binaries. The first time you are clicking on a binary, Windows will create this prefetch file and load some memory so that the next time you are executing it, it executes faster. That is quite useful for forensics, because it will record recent execution times for that specific process.
What are the limitations?
From experience something that is usually lacking from that data (like amcache shimcache, registry keys, and so on) is understanding how privilege escalation happened. What you can see is how the attacker moved laterally, some of the software that has been executed, the files that have been accessed on disk.
You might see the attacker using one account, and then connecting with an administrative account. So clearly there has been some privilege escalation there, but you can rarely get a glimpse into how.
Are there any specific tools you like to use?
I try not to stay attached to any tool at one time. They tend to change and evolve all the time. It’s more about feeling, staying excited for the field and the projects you are working on.
As an IR lead, how do you gain confidence in delegating analysis?
It’s about trusting your team. Trusting that they will come to you when they have questions or when they aren’t sure.
Encouraging them to do that and to ask as many questions as possible. I’d rather you ask me a hundred questions [at the start] than you not coming back to me with any update for two weeks, at which point it is probably too late.
Having a structured way of analysing things, like a list of artefacts that you as the response lead provide to people in advance. You are the main point of contact, you are scoping the incident. It applies to any kind of project, you are understanding what is needed and devising a plan of action that is then discussed and communicated to your team.
What changes are you are seeing in IR?
Full disk forensics is a lot less common that it used to be. I came into IR as the transition was being made from full disk forensics to collecting artefacts.
When analysing systems, you are relying on specific data sources that we call artefacts. Before cybersecurity tooling was a thing, you’d have to collect the whole disk, all your log sources would be extracted from there.
I remember the first time I had to do my first full disk forensic acquisition. I had to pick up about thirty laptops. Driving up to the client, getting chain of custody signed and completed. Putting laptops in evidence bags and making sure that the whole process is followed. That is the kind of thing you don’t want to mess up in any way.
You’ve mentioned collecting artefacts is more common, what do you think that drives that?
Nowadays, through the tooling and the fact that most servers are now hosted on cloud, the prevalence of SaaS [Software as a Service] etc. logs are just made available to you. All you need is a quick way to collect these.
What are the benefits of artefact collection?
From a consultants point of view, we have a duty to act in the interest of our clients. If that means collecting artefacts is possible rather than a full disk image, we should do that, so long as it doesn’t impact the accuracy of the data.
If I collect the data faster, that means I can focus more of my time on analysis. That means I can provide an update quicker to the client and maybe even the project will end up cheaper for the client.
If we are talking about endpoints, you can just use your XDR tool to just retrieve only the data sources that you need. That saves time, which is essential during incident response. It opens the avenue for automation, which is great. You don’t need to travel (to client site) as much. You just give the client the collector to run on their systems, and the data is back with you within hours.
This article was written following an open discussion. Responses capture the essence of the conversation, but may not be direct quotes and details are anonymised.