Hi all,
In the latest "Security Weekly" (https://www.youtube.com/watch?v=CXefYdEGW04
)
they present the Anthropological "Hacker" Map
https://wherewarlocksstayuplate.com/map/
While the map is incomplete (how can it ever be complete?), I think it is
one of the few times, outside of David Aitel's writings about the cross-cut
between the "underground" (for a lack of a better term) and subsequent
commercial activity.
Based on my personal bubble, completely empirical evidence that you should
take with a huge grain of salt the cross-cut is not the actual highlight
here - the highlight is a temporal one. The generation of folks contained
therein are the among the first two generations (or so) between the
commoditization of Internet - essentially having first movers advantage on
the space.
Anyway, felt like sharing this with the rest of the community here
I've seen great people in our industry crushed under the weight of the
secrets they carry into a singularity from which no information can emerge.
In some ways the lesson from apache_nosejob.c
<https://www.exploit-db.com/exploits/21560> was that we cannot take
ourselves seriously, that at the heart of our discipline there must remain
a jester, that we must float upon the stream of endless information rather
than absorb it into our darkened core.
To that end I often listen to infosec podcasts while doing other things:
1. The Three Buddy Problem
<https://securityconversations.com/podcast/security-conversations/> with
Ryan Naraine, Costin Raiu and JAGs. This is probably my new
favorite podcast, with an uncensored take on current infosec events,
largely from an incident response standpoint, but in general covering all
the bases and courageously offending everyone at great length.
2. Risky.Biz <https://risky.biz/RB775/>: Still excellent after all these
years, partially because Adam is such an experienced penetration tester and
Patrick is a good host, and I generally learn things about events that are
poorly covered in the news from their perspective, without having to go
through and do fact checking myself (aka, why the Struts bug is so bad,
etc.)
Also, happy New Year everyone!
-dave
[image: image.png]
It's impossible not to notice that we live in an age of technological
wonders, stretching back to the primitive hominids who dared to ask "Why?"
but also continually accelerating and pulling everything apart while it
does, in the exact same manner as the Universe at large. It is why all the
hackers you know are invested so heavily in Deep Learning right now, as if
someone got on a megaphone at Chaos Communication Camp and said "ACHTUNG.
DID YOU KNOW THAT YOU CAN CALCULATE UNIVERSAL TRUTHS BACKWARDS ACROSS A
MATRIX USING BASIC CALCULUS. VIELEN DANK UND AUF WIEDERSEHEN!".
Hackers are nothing if not obsessed with the theoretical edges of
computation, the way a 1939 Niels Bohr was obsessed with the
boundary between physics and philosophy, and how to push that line as far
as possible using math in such a way that you could find complementary
pairs of truths lying about everywhere and of course one aspect of his work
is that you can almost a century later deter an adversary from attacking
you by threatening to end the world, and another aspect is you can study
the stars and as a bureaucrat you would call this "dual-use" although
governments tend to have a heck of a lot of use for the former and almost
no use at all for the latter.
All of which is to say that while NO HAT <http://nohat.it> is a very good
conference, with a lot of great talks, the one I have enjoyed the most so
far is "LLMs FOR VULNERABILITY DISCOVERY: FINDING 0DAYS AT SCALE WITH A
CLICK OF A BUTTON <https://www.youtube.com/watch?v=Z5LMRS3AF1k>" (by
Marcello Salvati and Dan McInerney). This talk goes over their lessons
learned developing VulnHunter <https://github.com/protectai/vulnhuntr>,
which finds real vulns in Python apps given just the source directory, and
those lessons are roughly as follows:
1. Focus on Specific Tasks and Structured Outputs: LLMs can be
unreliable when given open-ended or overly broad tasks. To mitigate
hallucinations and ensure accurate results, it's crucial to provide highly
specific instructions and enforce structured outputs. (aka, the naive
metrics people are providing are probably not useful).
2. Manage Context Windows Effectively: While larger context windows are
beneficial, strategically feeding code to the LLM in smaller, relevant
chunks, like focusing on the call chain from user input to output, is key
to maximizing efficiency and accuracy. They did a great job here, and this
is important even if you have a huge context window to play with (aka,
Gemini).
3. Leverage Existing Libraries for Code Parsing: Dynamically typed
languages like Python present unique challenges for static analysis.
Utilizing libraries like Jedi, which is designed for Python autocompletion,
can significantly streamline the process of identifying and retrieving
relevant code segments for the LLM. They recommend a rewrite here
themselves using treesitter to look at C/C++, although I would probably
personally have used an IDE plugin to handle Python (and give you debugger
access).
4. Prompt Engineering is Essential: The way you structure prompts has a
huge impact on the LLM's performance. Clear instructions, well-defined
tasks, and even the use of XML tags for clarity can make a significant
difference in the LLM's ability to find vulnerabilities. But of course, in
my experience, the better your LLM (larger really), the less prompt
engineering matters. And when you are testing against multiple LLMs you
don't want to introduce prompt engineering as a variable.
5. Bypass Identification and Multi-Step Vulnerability Analysis: LLMs can
be remarkably effective at identifying security bypass techniques and
understanding complex, multi-step vulnerabilities that traditional static
code analyzers might miss. There's a ton of work to be done in future
analysis in how this happens and what the boundaries are.
6. Avoid Over-Reliance on Fine-Tuning and RAG: While seemingly
promising, fine-tuning LLMs on vulnerability datasets can lead to
oversensitivity and an abundance of false positives. Similarly, retrieval
augmented generation (RAG) may not be sufficiently precise for pinpointing
the specific code snippets required for comprehensive analysis. Knowing
that everyone is having problems with these techniques actually is good
because it goes against the common understanding of how you would build
something like this.
At its core, vulnerability discovery is as much about understanding as it
is about finding flaws. To find a vulnerability, one has to unravel the
code, decipher its intent, and even the assumptions of its creator. This
process mirrors the deeper question of what it means to truly understand
code—of seeing beyond syntax and function to grasp the logic, intention,
and potential points of failure embedded within. Like Bohr’s exploration of
complementary truths in physics, understanding code vulnerabilities
requires seeing both what the code does and what it could do under
different conditions. In this way, the act of discovering vulnerabilities
is itself a study in comprehension, one that goes beyond detection to touch
on the very nature of insight.
-dave
The Anatomy of Compromise
One of my demented hobbies is watching old infosec talks and then seeing
how well they hold up to modern times. Recently I excavated Metlstorm's
2017 BSides Canberra
<https://www.youtube.com/watch?v=OjgvP9UB9GI&list=TLGGvAY1CcIr-AcyNjEwMjAyNA>
talk on "How people get hacked" - a pretty generic topic that gives a lot
of room for opinion, and one a lot of people have opined on, but the talk
itself has a lot of original things to say. In particular, there's a huge
disconnect between how people get hacked and how defenders and policy
makers think people get hacked and choose to defend against them - which
anyone on this list already knows - I think we are all aware that defensive
strategies in cyber are rarely based on available data.
[image: image.png]
The Three-Act Play of Compromise
Here is how people get hacked, according to Metlstorm:
1. Find something with 1FA and crack it open (or just phish the creds)
(Everything in "Secure By Design" is meant to address this part of the
problem)
2. Get Domain Admin and hang onto it
3. Watch the person who does the important stuff (like SWIFT transfers)
and secretly do their job for them
Metlstorm goes into the Active Directory hacking that we all know and love
in great detail. His toolbox from 2017 (Kerberoasting, Group Policy files,
password spraying, etc.) is still largely relevant today, despite Dwizzle's
best work - and points out that removing an attacker that has once had
domain admin is practically impossible even though we all pretend it is to
the SEC (a painful truth we don't deal with at all in industry, unless Wiz
has a product line here I don't know about).
But the pattern he's really describing is the understanding that individual
vulnerabilities and Active Directory "features" are as relevant to systemic
compromise as individual genes are to having an arm with five wiggly bits
at the end. Metlstorm picks on Active Directory and its cousin Sharepoint
quite a bit, but his point is not that we should blame Active Directory so
much as ourselves - we all installed something huge and complex we didn't
understand and then put the keys to our kingdoms in it.
Partially he doesn't blame AD because Metlstorm, even before the SolarWinds
and Kaseya compromises happened, was obsessed with supply chain weaknesses
- or rather he clearly looks at it not as a supply chain but a supply web,
where compromise propagates through trust relationships like signals
through a neural network.
And this is where Metlstorm's talk becomes particularly interesting in
retrospect. While we were all obsessing over Domain Admin and Exchange bugs
in 2017, he was pointing at MSPs and software providers saying "that's
where the real action is." In the years since, we've seen exactly this
pattern play out in increasingly sophisticated ways:
- SolarWinds and Kaseya (2020-2021) showed us what happens when
attackers compromise either a build pipeline or an MSP's distribution system
- Recent MSSP breaches that none of us will ever hear about unless the
GCSB decides to write them up
Each of these compromises followed Metlstorm's basic thesis: why hack 1000
companies when you can hack the one company they all trust? The attackers
don't see individual organizations - they see connection points, trust
relationships, and privileged channels that can be repurposed. Seven years
later, this view has proven devastatingly accurate.
Metlstorm calls himself an "operational hacker" - different from your Brett
Moore style "Research Hacker" who's all about finding bugs and writing
shellcode and various useless stuff like that. For him, operational hacking
is about systems thinking: what does each compromise actually get you? And
this, as it turns out, is what the talk is really about.
Digital Ecosystems
[image: image.png]
Using New Zealand as his laboratory, Metlstorm somewhat cheekily shows us
organizations not as isolated entities but as nodes in a vast supply web:
- Managed service providers spreading their digital mycelia through
thousands of organizations
- "Liz in accounts payable" unknowingly holding the keys to national
security
- Domain registrars running code old enough to be geological
This is one of the strengths of the talk - it is backed up by specifics. It
is not a vague thought-piece. He takes shots at the whole "I hunt
sysadmins" approach as thinking too small! Why hunt sysadmins when you can
hunt their managed service providers who already have domain admin? Or,
hunt the providers of those providers. It's like a food web of sysadmins.
His best examples are massive US companies (NYT, f.e.) that got owned
through tiny companies in NZ- big for NZ standards maybe, but microscopic
globally.
The Observer Effect
What Metlstorm as an attacker sees everywhere he looks is large systems
that are "commercially untestable" - creating a fundamental disconnect
between risk and reality. When you outsource your domain admin to a global
megacorp (or your local Kiwi-buds), you create a quantum state of security
- simultaneously compromised and secure until someone attempts to measure
it.
You:
- Can't test their security
- May not know if they're compromised
- Certainly can't perform incident response
- But get a lovely compliance certificate to frame
Recent compromises prove what Metlstorm saw in 2017: while defenders obsess
over hardening their membranes via the magic of secure by design (or paying
"$6 a month for MFA"), attackers traverse the supply web and pick on
whatever provider seems easiest to own.
The reality is that no organization exists in isolation any more than a
neuron functions alone. Your security isn't just your controls - it's every
provider, vendor, and service in your supply web, each one a potential
firing synapse of compromise.
*Solutions*
[image: image.png]
He rightfully calls out that we will not solve these problems. So an A for
Accuracy. Very fun talk, worth your time, highly recommended, 10/10 would
listen to again in the car on the way to a house built at sea level in a
hurricane zone.
-dave
I spent some time watching all the Grace Hopper videos on the youtubes, as
I prepared for what up North is a horrible storm, but here in Miami is, so
far, a breezy and clear day. You can hear her talk about how subroutines
used to be literal handwritten pages of instructions in notebooks. When you
wanted SIN or COS you would go over to whoever had the notebook with the
working version, and copy it out into your code.
It was this experience that guided her as she wrote the first compilers,
but as you listen to her stories you can see her ponder the meta-questions
of computation as well - she uses as the example how a game of basketball
sparked her solution to a troublesome issue in her first single pass
compiler, leading to her writing what must have been the first jump table.
How, she asked herself, can we get a computer to capture this geometry of
the mind. What was it that made her brain think of jump tables while
playing basketball?
When asked about AI, she poo-pooed theoretical work, and emphasized
practical needs: data processing and expert systems for field commanders.
Hopper's journey from abstract thought to concrete application mirrors our
ongoing struggle in computer science, as does her struggle to get the
military to adopt new technology, which she relates with anecdote after
anecdote - clearly the hardest thing she did in her career was get support
for computers into the Navy bureaucracy.
This year I've attended a birthday and a funeral, and a lot of our
community was at both. But there hasn't been a good offensive-minded
conference for us to attend in the States after INFILTRATE ended. We've
needed one. Our collective knowledge, like Hopper's compiler, builds upon
itself, each pass adding depth and functionality.
Two upcoming conferences, RE:VERSE in Orlando and District:CON in DC, have
now arisen to continue this tradition. They're not just tech showcases, but
gatherings of minds, a hive of soldier ants collected around a temporary
bivouak, each attendee contributing to our shared codebase of ideas, as if
scribbled in a 1950's coders notebook. We're still trying to capture the
geometry of thought, one exploit at a time. And I hope to see everyone
there!
-dave
Grace Hopper video (best version imho because it includes Q&A - link is to
the AI question):
https://youtu.be/WyGtNvBZ6kk?si=aF4b3elx2vDPPNax&t=5054
Conferences (please buy tickets now before they run out and I am sad):
1. https://re-verse.io/ - Feb 28th in Orlando, FL
2. https://www.districtcon.org/ Feb 21 in Washington DC
As you know, humans like to invent comfort words. One of my favorites is
"luck". The theory being that yes, the universe has dice, but they are
loaded in your favor. Properly used, these words are a spell - they allow
us to have courage when a sober mind would quail. But when you become a
professional, you have to give up these crutches. Only poor poker players
believe in "luck".
In computer science, and especially in machine learning, one of these words
you have to give up is "consciousness", which makes a lot of the current
discussion on AI feel pompous and unbearable. Just because you can feel
something doesn't make it real. The other day someone said to me, "I don't
know why, but it FEELS true to me" and I haven't been able to get that out
of my head since then.
In security we are often challenged to reject the assumptions a system is
built on so we can develop a third sight - in some cases clear enough you
gain the ability to predict the future. The raw fuel of this is a vast
reliance on an information torrent that to any outside party looks like
addiction.
I'm by no means an expert on war, conflict, or the Middle East, but what
Israel has done to Hezbollah is stunning to most of the experts, and in my
opinion, it's because they missed a major shift in warfare that was largely
driven by *Java Middleware*, of all things. So many of us lived through the
transformation brought about by precision GPS and ISR-based air power. This
much is obvious to the crowd that writes articles in war journals. Air war
is cool in a very teenage way, and drone movies have a certain visceral
feel to them that academics like to write about because it says to the
world that they are the kind of hard-core experts that understand the
blood-and-guts reality of modern combat. You know what's not sexy? Java.
But what is network centric warfare
<https://en.wikipedia.org/wiki/Network-centric_warfare#:~:text=Network-centr…>
really if not endless rooms full of programmers having meetings about Java
Beans and SOAP contracts. What does it look like when a real team goes
after your entire society and builds surveillance into everything? It's
hard to move in the modern world without a wake billowing behind you into
the ether. Al Quada learned this the hard way, and now I think we can see
what it looks like applied to another decentralized covert organization.
In the end, it's not the drones or the missiles that make the
difference—it's the quiet, relentless infrastructure underneath it all,
overlooked by those too focused on the shiny surface of modern warfare. The
real revolution isn’t in the explosions but in the code. The experts who
missed this shift weren’t outmaneuvered by luck or brute force; they were
outmatched by those who embraced the unglamorous, intricate work of
building a machine that sees all and remembers everything. The future of
conflict, like security and AI, won’t belong to those dazzled by the
spectacular. It will belong to those who understand that, more often than
not, the real power lies in the invisible.
-dave
People doing software security often use LLMs more as orchestrators than
anything else. But there's so many more complicated ways to use them in our
space coming down the pipe. Obviously the next evolution of SBOMs
<https://www.cisa.gov/resources-tools/resources/cisa-sbom-rama> is that
they represent not just what is contained in the code as some static tree
of library dependencies, but also what that code does in a summary fashion
that you can check once you get the final binaries. In a certain sense, you
can think of this as a behavioral attestation between the software
publisher and the consumer who is actually running the product.
In other words, if my product is meant to connect to WWW.SPYWARE.RU, then
it should say so in the SBOM behavioral manifest. But of course in practice
these things get quite complicated, and hence you need to summarize
semi-structured data (aka, the behavioral manifest is rarely exact), and
then compare it to what is seen when the software itself is run (which if
you've ever run strace ...is voluminous). That smells like a job for an
LLM, or at the very least, a vector comparison. Likewise, automatically
building harnesses to run and capture security sensitive information (or
performance information as we learned from XZ), is rapidly also becoming a
job <https://google.github.io/oss-fuzz/research/llms/target_generation/>
for an LLM.
I perhaps am channeling everyone else's
<https://www.cisa.gov/speaker/allan-friedman> worry that too much of the
SBOM community is arguing about which XML fields belong in a VEX addendum,
rather than pushing the concepts forwards to actually solve problems. Or
perhaps not! At some level, the software vendors are getting dragged
through this process by their hair, which is very fun to watch.
-dave
Before there were words, calculated as the softmax of a list of possible
tokens, there were just vectors of nano-electrical potential in cells
soaked in a hormonal brew of electrolytes, operating on a clock cycle of
"slow, but fast enough". In this sense, as we now know
<https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10472538/>, we generate words
and we know, in our heads, what we are, in the same way as we generate
limbs, with each cell knowing from its electric field what to be next. A
tumor is in that way of thought a *confabulation *or as we now say, a
*hallucination.* But then, also, so are you.
Recently I spent some time reading this year's Research Handbook on
Cyberwarfare
<https://cybersecpolitics.blogspot.com/2024/08/a-quick-research-overview-of-…>.
One of the forms I filled out recently asked me if I was a certified Master
Operator, which of course, I am not, any more than an Archaeopteryx is a
certified Bald Eagle, even though both know the smell of the sky and the
taste of freshly caught fish. But I do occasionally pay attention to the
"state of the art" academic view of cyberwar and the Handbook was a good
way to catch up.
For example if you read Nadiya Kostyuk and Jen Sidorvova's Handbook paper
on *Military **Cybercapacity* they will say that "a cyber attack may
provide a defender or third party with a good estimate of the attacker's
capabilities, but it is not clear how many of these capabilities the
attacker has in their arsenal". This is, to my primitive cyberwarfare mind,
so old that I still use "screen" instead of "tmux", a bit of a misstep when
it comes to how cyberwar works and what a capability is. I don't know how
to say it any clearer than this: Behind every wooden horse is a woodshop.
An example in my head is that right now the Ukrainian army is rumored to be
sitting on top of a major gas terminal in Kursk, one responsible for
supplying Russian gas to Europe. You have to assume that, having learned
from the Russian attacks against their electrical infrastructure, the
Ukrainian Army is traveling not just with a screen of FPV drones but with a
few USB keys containing implants for the specialized equipment that runs a
gas network.
It's hard to disconnect OT networks that are presumed to be segmented
physically, and temporary physical control can easily translate to
permanent cyber control. And cyber control, despite what Quentin E.
Hodgson's Handbook paper (*Cyber coercion as a tool of statecraft: how
often, how effective?*) wrongly concludes, is extremely useful for state
coercion.
Perhaps the problem with the Handbook, like all academic writing on
cyberwar, is that it is meant to be sterile. But that's not how cyberwar
works, held in the space that is a melange of electrons and intentions. As
tumors confabulate within flesh, so too do our digital dreams hallucinate
new worlds, both the virus and the firewall, the wooden horse, and the
workshop that births it. Certified or not, we are masters of a domain we
cannot fully comprehend, sailing on seas of raw data, guided by stars we
ourselves ignite.
DefCon is a study in cacophony, and like many of you I'm still digging
through my backlog of new research in multifarious browser tabs, the way a
dragonfly keeps track of the world through scintillated compound lenses. In
between AIxCC (which proved, if anything, the boundaries
<https://dashboard.aicyberchallenge.com/collectivesolvehealth> of automated
bug finding using current LLM tech?), James Kettle's timing attack research
<https://portswigger.net/research/listen-to-the-whispers-web-timing-attacks-…>,
and even more PHP ownership
<https://www.ambionics.io/blog/iconv-cve-2024-2961-p1>, you unfortunately
do have to pay attention to the outside world.
One of the things that lit up my sensors was the Windows Remote Desktop
Licensing service that came out from a sort of "Post QiHoo360" exploit
community, led by Dr. Zhiniang Peng (aka @edwardzpeng), an absolute legend
of exploitation. A remote unauthenticated heap overflow in the latest
Windows via an MSRPC endpoint, bypassing modern defenses by just calling
LoadLibraryA("\\webdav\owned.dll") on a fake object. An unexpected burst of
pure beauty really, like the iridescence of a Morpho moth flitting across a
concrete parking lot. The exploit
<https://github.com/CloudCrowSec001/CVE-2024-38077-POC/blob/main/CVE-2024-38…>
is public, but the original paper is now mysteriously deleted, I assume for
political reasons. If you have a copy of it, please shoot it my way. It's
telling that all the best exploits I know have "Exploitation less likely"
as their rating from Microsoft.
Anyways, it's interesting what merits attention, and what doesn't.
-dave
Cordyceps Analysis Report on PRANA Network Hack and Leak Operation:
https://docs.google.com/document/d/1oOJbBTUwyK85ZKYAAdwWqxk-sMvqrBqzJYX1ozi…
Lately I've been reading a lot of academic papers, mostly the Research
Handbook on Cyberwarfare
<https://www.elgaronline.com/edcollchap/book/9781803924854/book-part-9781803…>.
Some of them are good papers! JD Work has a paper in it! But also some of
them get wrapped around the idea of "Cyber War vs Cyberwarfare" (!??!) or
fall in love with "hacktivists" or stomp furiously around all the real
issues in the domain without ever stepping on the green.
The thing about Persistent Engagement and Integrated Deterrence and Defend
Forward is that yes, as Paul Wither's points out in "Do we need an
effects-based approach for cyber operations?
<https://kclpure.kcl.ac.uk/portal/en/publications/do-we-need-an-effects-base…>"
or as the Grugq points out in his latest keynotes
<https://www.youtube.com/watch?v=P6PnhDfWvx0>, we do need to look at
"effects" in a broad way when talking about cyber operations. But the goal
of your offensive counter-cyber operations is not to "reduce the number of
incoming operations", as Jay Healey would try to measure, but to mitigate
strategic effects!
In other words, if the Chinese are hacking as much as possible in the LLM
space, but somehow don't manage to launch their own market-leading LLM
because for some reason their experiments on Huawei Ascend chips all
mysteriously have weirdly wide bands of error, that is Defend Forward
working! Did nobody in all of cyber policy academia read the Three Body
Problem?
Academics should love this part of the theory, really, because arguing
about strategic effects is 90% of what they do, usually without ever
opening up a Neo4j Database filled with stolen mail spools to see what's
really happening on the wire.
Anyways, if you like the kind of reporting in our PRANA report, we offer
paid reporting as well. :)
Thanks,
Dave Aitel