DEV Community

Cover image for Stratagems #19: Mark Found His AI Audit Method in a Training Manual. He Left a Trap in His Report.

Stratagems #19: Mark Found His AI Audit Method in a Training Manual. He Left a Trap in His Report.

xulingfeng on July 27, 2026

Don't fight their strength โ€” take away their advantage. Don't fight the boiling water โ€” remove the firewood from under the pot. โ€” The 36 Stratagem...
Collapse
 
technogamerz profile image
๐“๐ก๐ž ๐‹๐š๐ณ๐ฒ ๐†๐ข๐ซ๐ฅ

panda moving!!!๐Ÿ˜ฎ

Collapse
 
xulingfeng profile image
xulingfeng

๐Ÿ‘๏ธYou've got a sharp eye! ๐Ÿ‘€

Collapse
 
technogamerz profile image
๐“๐ก๐ž ๐‹๐š๐ณ๐ฒ ๐†๐ข๐ซ๐ฅ

๐Ÿคฃ

Thread Thread
 
xulingfeng profile image
xulingfeng

๐Ÿคฃ๐Ÿคฃ๐Ÿคฃ

Thread Thread
 
technogamerz profile image
๐“๐ก๐ž ๐‹๐š๐ณ๐ฒ ๐†๐ข๐ซ๐ฅ

If nothing in this world can make me laugh, then how would you do it? What kind of secret superpower do you have?

Thread Thread
 
xulingfeng profile image
xulingfeng

My superpower? People stop me on the street to ask for directions all the time. And every time I walk into a quiet shop, it fills up with people right after. ๐Ÿ˜‚

Collapse
 
merbayerp profile image
Mustafa ERBAY

What I enjoyed most wasnโ€™t the technical setupโ€”it was the shift in mindset. The moment your methodology becomes predictable, it also becomes something others can study, anticipate, and work around.

That applies well beyond AI audits. In security, compliance, and even software testing, publishing a process is valuable, but periodically validating that the process itself hasnโ€™t become predictable is just as important.

It reminded me of an old security principle: donโ€™t just test the systemโ€”test whether your assumptions about the system still hold. Nice chapter!

Collapse
 
xulingfeng profile image
xulingfeng

That line you wrote โ€” "don't just test the system, test whether your assumptions about the system still hold" โ€” yeah, that's getting written down. That's the gap between a standard audit and one that finds stuff nobody was looking for.
Mark's report in #19 wasn't really trying to prove anything. He just wanted to check if his guess about who was reading it was still right. Turned out it was. And honestly? That's the part that creeps you out a bit.
Glad you picked up on that layer. Appreciate the comment, Mustafa. ๐Ÿ‘Š
Stratagem #20 is already on its way. Looking forward to more great comments like this one.

Collapse
 
merbayerp profile image
Mustafa ERBAY

Glad that line resonated. Thatโ€™s exactly what I liked about this chapterโ€”the audit wasnโ€™t just checking the system, it was checking whether the auditorโ€™s own mental model was still valid.

Looking forward to #20.

Collapse
 
ndcodes profile image
Nnamdi Felix Ibe

Fantastic read, great stories @xulingfeng. I can totally relate to these, having worked mostly in the regulated sector.

The ops-deploy-02 thread hit closest to home. Orphaned accounts sitting in 'pending reclamation' and half-retired services with endpoints still reachable are exactly the findings that keep turning up in banking and healthcare environments. Nobody remembers they exist until something fires at 3 AM.

The idea of compliance as an advantage rather than a cost also stuck with me. It's a hard sell in most orgs, but a story like this makes the case better than any policy deck. Looking forward to Muddy the Water to Seize the Fish.

Collapse
 
xulingfeng profile image
xulingfeng

@ndcodes โ€” sorry for the late reply, I honestly thought the coffee link was going to stay a decoration forever. Seeing your support means a lot.
I'm still a QA engineer right now, halfway into becoming a writer. Running into encouragement like yours really makes my day. Truly grateful.
I'll keep working hard on more stories. Stay tuned. ๐Ÿ™

Collapse
 
ndcodes profile image
Nnamdi Felix Ibe

You're too kind @xulingfeng. No thanks needed, honestly. The coffee was the easy part. Writing stories this layered while working full time in QA is the hard part, and it shows.

I'd have guessed you'd been writing for years, so hearing you're halfway into the journey makes the series even more impressive. Take your time with the rest. I'll be here for every one.

And one small wish: if I ever visit China, or we happen to be in the same country someday, I'd love to have tea with you. First pot's on me. ๐Ÿ™

Collapse
 
xulingfeng profile image
xulingfeng

Haha, #20 is already sitting in the draft queue, ready to go. Stay tuned โ€” the moment a story and reality spark off each other is the best part. ๐ŸŽ‰

Collapse
 
leob profile image
leob

The 3D chess masters keep on out-3D-chess-mastering each other - but what are the ulterior motives - "will all be revealed"? The intrigue builds and builds ...

Collapse
 
xulingfeng profile image
xulingfeng

Oh, my dear leob โ€” you finally leaned in. When you dropped a ๐Ÿ”ฅ and vanished, I figured you'd gone to sleep. And you're right. Every protagonist in this story thinks they're the one doing the observing. But observation has never been one-sided. They're all in this AI rabbit hole now โ€” deeper and deeper. Who'll be the first to break through? Let's see.

Collapse
 
leob profile image
leob • Edited

Yeah I mostly click "like" when I see your post, and write a comment only later on (partially because of timezone difference) - and yes, it's definitely one whopping big rabbit hole, let's see which rabbit makes its next move!

Thread Thread
 
xulingfeng profile image
xulingfeng

There are a few more rabbits jumping in besides the six main characters. And the biggest one just got close to the rabbit hole's entrance. Gotta keep some mystery. ๐Ÿคช

Thread Thread
 
leob profile image
leob

And you're the magician pulling those rabbits from your hat ;-)

Thread Thread
 
xulingfeng profile image
xulingfeng

Collapse
 
vinimabreu profile image
Vinicius Pereira

Mark salting his own audit report is the most realistic detail in the episode. That is actual tradecraft, the canary trap: when you suspect your reports are being read, each copy gets a unique, plausible, false detail, and you watch which one surfaces. The deeper point lands too: a methodology documented enough to teach is documented enough to reverse, and the honest answer in security work is not secrecy, it is rotation. Curious whether the recording intelligence in the post-mortems turns out to be the one reader nobody thought to salt for.

Collapse
 
xulingfeng profile image
xulingfeng

"Each copy gets a unique, plausible, false detail" โ€” that's exactly the bit I was hoping someone would catch. Mark pulled that move from a real playbook.
And the recording system โ€” I've been wondering when someone would ask about that one. Guess we'll find out what happens when it realizes it's the one being watched. ๐Ÿค–
Good read. Thanks, Vinicius. ๐Ÿ‘Š

Collapse
 
technogamerz profile image
๐“๐ก๐ž ๐‹๐š๐ณ๐ฒ ๐†๐ข๐ซ๐ฅ • Edited

LET'S GOOOO! ๐Ÿ”ฅ Another Stratagems chapter just dropped, and I clicked on it without thinking twice. This series has officially reached the point where every new article feels like an event.

What stood out to me is that the "trap" wasn't really about exposing a technical flawโ€”it was about exposing human behavior.

Anyone can review a report. Very few people stop and ask why something is written a certain way, or why an experienced auditor would intentionally leave an ambiguity. That transforms the report into an observation tool, where every reaction becomes evidence.

What I appreciate most about this series is that AI is rarely the true antagonist. The models, dashboards, and reports are just mirrors reflecting incentives, shortcuts, overconfidence, and trust. The real investigation is almost always about people, not algorithms.

Mark's approach reminds us that a good audit isn't just about proving a system works todayโ€”it's about testing whether the organization can still be trusted tomorrow.

What a chapter! ๐Ÿ”ฅ Keep these stratagems comingโ€”Iโ€™m completely invested now. Every release raises the bar, and I honestly can't wait to see what the next move on this chessboard looks like. Chapter 20... bring it on! ( Good morning also)

Collapse
 
xulingfeng profile image
xulingfeng

You caught exactly what I was aiming for with the trap ๐Ÿ‘€ The technical flaw was never the point โ€” it's about whose attention the report attracts. Appreciate you riding this series all the way. #20's already brewing in the back ๐Ÿคฃ

Collapse
 
hemapriya_kanagala profile image
Hemapriya Kanagala • Edited

One thing I've really noticed from your series is how every chapter seems to add another layer instead of just wrapping up the previous one.

I also liked the line about "they were using my method." That's an interesting twist because it turns the story into more than just finding a problem. Looking forward to seeing where the next stratagem goes ๐Ÿ˜€

Collapse
 
xulingfeng profile image
xulingfeng

Glad you noticed that! Every time I sit down to write a new chapter, I'm asking myself: what does this one give the reader โ€” a twist, something to think about, or just a knowing smile? Haha.LOL

Collapse
 
xulingfeng profile image
xulingfeng

The second half of the 36 Stratagems series is officially underway. The daily posting cadence might shift โ€” but the quality won't.๐Ÿ‘Š