Valid JSON can still be wrong
There are two gates between a model’s output and your workflow. Most teams only build the first one.
Hi there!
You’ve got an LLM turning show interface output into JSON. It works across your test samples, the fields come back clean, and you wire it into the next step.
Now give it this:
GigabitEthernet0/1 is up, line protocol is down (err-disabled) If the model returns a neat JSON object describing a healthy interface, json.loads() will accept it without complaint.
Nothing crashed. That’s the problem.
Model output must clear two gates, not one
When json.loads() succeeds, you’ve cleared the first gate: the response is syntactically valid JSON.
That tells you nothing about whether the content is correct. Valid JSON can still:
Omit a required field
Return an MTU as text instead of an integer
Return a state your schema doesn’t allow
Invent a value that never appeared in the source output
The second gate is validation, and it belongs in your code.
Sif Baksh draws this boundary clearly in Building AI Agents for Network Operations: the model helps transform the text, but the application owns the trust decision. Until your code validates it, an LLM response is only candidate data.
First ask whether you need the model
If the device returns native JSON, use it.
If a reliable TextFSM template already exists, use it. If the format is stable and under your control, a deterministic parser will usually be faster, cheaper, and easier to test.
LLM-assisted parsing earns its place in a narrower band.
The clearest example is multi-vendor normalization. Cisco IOS might return GigabitEthernet0/1, Arista Ethernet1, and Juniper ge-0/0/1.0. The text differs, but you may want each result mapped into the same shape:
{
"interface": "...",
"admin_status": "...",
"oper_status": "..."
} The other good fit is text that was never structured to begin with: incident notes, ticket bodies, or mixed log output containing prose.
Use the model where flexibility helps. Keep exact decisions in deterministic code.
Validate the smallest useful shape
Start by checking whether every required field exists:
expected = [
"interface",
"admin_status",
"oper_status",
"ip_address",
"mtu",
]
missing = [
field for field in expected
if field not in result
]
if missing:
print(f"Missing fields: {missing}")
else:
print("All expected fields present!") This is only the first validation check. A production workflow should also confirm:
Each field has the expected type
Status values come from an allowed set
Values agree with the source data where possible
Missing or unsupported values are handled safely
A typed data model or validation library such as Pydantic makes these checks easier to enforce before anything downstream touches the result.
The key phrase is before anything downstream touches it. Logging a validation warning while still passing the object forward defeats the point.
Validated output is only one part of a safe workflow. If the next step is an infrastructure change, policy enforcement and approvals need to live in the execution layer, not in the prompt.
Turn infrastructure intent into governed execution
Stop managing IaC manually. Spacelift brings AI-native orchestration to Terraform, OpenTofu, Ansible, and more. With automatic drift detection, policy as code, approval workflows, and natural-language provisioning through Spacelift Intelligence, teams can accelerate infrastructure delivery while retaining the governance and visibility production environments require.
Give uncertainty somewhere to live
Real device output doesn’t always fit neatly into up or down.
An interface may be err-disabled. A port may be flapping, leaving its current state unclear. A command may return % No interface information available.
If your schema only permits clean states, the model has two choices: break the schema or guess. Models are very good at choosing the second option.
Allow states such as unknown, and include fields such as error_reason and warning. Keep those fields empty unless the reason appears in the source data. A failed command should return a controlled error or an explicitly uncertain result – never a confident, invented status.
The same rule applies when the model produces malformed output. It may wrap JSON in Markdown fences or add a helpful sentence before the object. Strip the common wrappers and retry. If it still won’t parse, return a structured error and let the caller decide what happens next.
Never fail silently, and never pass a half-valid result forward.
Try this on one parser
Take one parser you already use (or the next one you’re planning), and check whether it does these five things:
Requests only the fields the workflow actually needs.
Uses a low temperature for predictable output.
Checks required keys, types, and allowed values.
Represents missing or uncertain data explicitly.
Stops the workflow when validation fails and logs the raw input, model response, and parsed result.
That last check pays for itself the first time something breaks during an incident. You need to know whether the device returned unexpected data, the model produced a bad response, or your code accepted something it should have rejected.
This issue draws on Building AI Agents for Network Operations by Sif Baksh, out now from Packt. Chapter 4 is the one to read if you’re building parsers. The labs run on local models via Ollama with mocked device data, so you can work through the whole thing without touching a production device.
Agentic RAG for Network Operations - Tuesday, August 25th, 9 AM EDT
Sif Baksh, Principal Solutions Architect at Tines, himself is teaching you how to build a RAG-powered NetOps assistant that pulls answers from your own runbooks, device configs, and troubleshooting notes instead of the open internet, so it can tell you why a BGP neighbor is stuck in Active, with the actual source cited. You’ll also learn the guardrail patterns that stop the assistant from inventing answers or recommending unsafe production changes, so it’s something you can trust mid-incident.
This one’s capped on seats and going quicker than usual. Use code LIMITED40 for 40% off.
P.S. - We have a special bundle for you where along with the event you get access to the book. Avail now
Aug 27th · Agentic AI for Infrastructure Engineering: From Chatbots to Operators
Ritesh Vajariya, founder & CEO - AI Guru, teaches you how to build an infrastructure agent that reads pod events, correlates live metrics, and proposes fixes it only executes once you approve, replacing the documentation chatbot most platform teams already outgrew. Seven production-grade failure scenarios, injected into your own local Kubernetes cluster, yours to keep and re-run after the session ends.
If your Docker and Kubernetes fundamentals need shoring up before you get there, The Ultimate Docker Container Book (4th edition) by Dr. Gabriel Schenker is the deepest single resource we carry on containers through orchestration, and its latest edition adds AI-driven DevOps patterns on top.
Aug 29th · Active Directory and Entra ID in a Modern Hybrid Architecture
Professor Robert McMillen breaks down how traditional Active Directory and Entra ID work together in real hybrid environments: domains and group policy on one side, cloud identity and SSO on the other, and Entra Connect bridging them. Built for IT pros moving into infrastructure or identity-focused roles.
Since the session is already an add-on to the Azure basics going in, Microsoft Azure Fundamentals Certification and Beyond (built around the January 2026 AZ-900 update) is worth having on hand beforehand, especially if you’re eyeing the certification alongside the hands-on identity work.