This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.
I spent a week deploying a CrewAI agent to AWS Bedrock AgentC...
For further actions, you may consider blocking this person and/or reporting abuse
The missing logs before initialization issue is such a pain point. Aside from Sentry, did you find a clean way to direct standard error/stdout to CloudWatch during the container initialization phase before Python starts up?
Honestly no not really. The problem is AgentCore doesnt even create a log stream in cloudwatch until your app process actually writes something to stdout
after init. So if it dies before that you get literally nothing what I started doing after this whole mess is wrapping the entrypoint in a shell command that prints basic stuff before python starts
dockerfile
ENTRYPOINT ["/bin/sh", "-c", "echo container starting uid: $(id -u) && exec python -m my_agent"]
That way if it crashes on the uid thing you at least see the mismatch Its not pretty but would of saved me like 4 hours honestly
Brilliant idea. Simple, but it definitely solves that maddening 'black hole' logging behavior before the process spins up. Appreciate the tip!
Yes Thanks you!
Stay tune for the next article
dev.to/sarvar_04/sentrys-span-hier...
This is the gap between reported success and observed success. Builds, deployments and HTTP status codes are claims. BootProof exists to make software produce evidence: start the real system, exercise its declared readiness contract, validate the response, and sign what actually happened. No proof, no green check.
boot-proof.com
Thanks for sharing 🙏🏻
The throughline here is that the real failures were contract failures, not just bugs: package source ambiguity, success-shaped error responses, container runtime assumptions, and naming rules that only exist in rejection behavior. The 200 OK that still means failure is especially nasty because it trains teams to trust the wrapper status instead of the payload that actually explains the break.
We have seen the same thing in agent and platform workflows. If the receipt only preserves the outer status and not the rejected body, missing directive, or runtime contract that failed, the next engineer has to rediscover the whole trail from scratch.
Curious whether you ended up automating any preflight checks for those undocumented constraints before the next deploy.
Yeah contract failures is exactly what it was the system never broke http semantics technically it just gave you back a response that was completely useless with no signal about what went wrong.
For preflight I ended up writing a small deploy script that checks 3 things before pushing:
nothing fancy but catches the exact 3 failures i ran into the naming regex one i just hardcoded as a check on the string. Honestly wish AgentCore had some kind of dryrun flag that validated all this server side before you wait 5 minutes for a deploy that silently fails
The 200-OK-on-every-error part is the one that would have cost me a full day too. When the transport layer swallows the real status, I have started logging the raw response body before any SDK deserialization, because that is usually the only place the actual failure shows up. Did the AgentCore container give you anything at all in stdout, or did you end up reading the SDK source to reconstruct why it crashed?
Nothing zero stdout The container would start, pass health checks for about 30 seconds, then just die no log stream even got created in CloudWatch which told me it was crashing before the logging framework could initialize. I ended up reading the sample Dockerfiles in AWS's GitHub repos and that's where I spotted the USER 1000 directive mine was missing. For the empty 200 problem, yeah I wish I'd done what you're doing logging the raw response body before deserialization. The SDK was calling response.json() on an empty string and quietly returning None, so by the time my code saw it the failure was already gone. Would've saved me a solid 5 hours if I'd just printed response.text before that call.
Great write-up. Nothing wastes engineering time quite like a system that fails successfully. A
200 OKwith an empty response is not observability, it is gaslighting.:::
Thanks Edgar!
Stay tune for next detailed article....
This is Intresting
Thanks You!