Behind

Setting Up Dev & Ops Environments for Our Agents (No Top-Secret Clearance Needed!)

Today, we’re peeling back the curtain to show you how our team sets up and leverages an agentic coding environment to build our AI video editor. In the fast-paced world of AI, workflows shift dramatically every 3 to 6 months (yes, blink and our stack changes!). While this post might have an expiration date, as long as the LLM paradigm holds, we thought it would be fun—and genuinely useful—to document our exact setup as of September 2026.

The biggest mindset shift? We no longer treat AI agents as just "another handy tool." Instead, they have become our primary interface—the starting point and the finish line for almost everything we do. To make this work, the real game is setting up accessible channels so AI can navigate and touch every piece of the pipeline: dev & testing, documentation, restricted DB access, logging, monitoring, and even customer support. It’s all about creating an environment where agents can be assigned tasks and managed seamlessly.

Going Remote: Dev Servers & CLI Workflows

Our code and CLI tools live comfortably on a remote development server, allowing us to jump in via SSH from literally anywhere—be it a high-powered laptop or a tablet at a coffee shop. Access is secured through a Tailscale mesh network, meaning our workspace isn't tied to a single local machine; we get the exact same repo and runtime environment wherever we go. Running tools like Claude Code (aka claude) or Codex CLI (aka codex) directly on the server lets us verify code changes instantly. Plus, with a 24/7 server running, setting up scheduled jobs, repetitive tasks, and continuous background automations is a breeze.

Letting the Agent Take the Wheel (Responsibly!)

Skipping constant confirmation prompts to let commands run sequentially—often dubbed "bypass permissions" or affectionately known as YOLO mode—is a game changer. It’s incredibly handy in a remote environment for speeding through low-risk, well-defined tasks like repetitive grepping, test suites, and log lookups without babysitting every keystroke.

Of course, with great YOLO power comes great responsibility. Letting an agent run loose expands its operational scope, and if it has access to production or critical credentials, a tiny hallucination can turn into a big headache. That’s why we bake in guardrails: pre-execution hooks to block sensitive files, mandatory confirmation checkpoints before touching AWS infrastructure, and strictly isolated Cloudflare credentials. Streamlining approvals is all about speed—not about giving away the master keys blindly!

Supercharging File Search in Claude 🚀

While we might not reference files manually as often as we used to, seamless file search is still an absolute lifesaver. In our daily workflow, we noticed that while Codex nails those clean, concise paths right away, Claude’s default file suggestions felt… a bit sluggish and clunky. So, we took matters into our own hands! We hooked up a custom script to `fileSuggestion` in `.claude/settings.json`.

This nifty script relies on `git ls-files` to scan only tracked files, prioritizing clean filenames and short paths. Type nothing? It smartly brings up your most recently edited files. It surfaces your project files at lightning speed compared to the default setup! Just keep in mind: untracked files won't show up in suggestions, and you'll need external dependencies like `fzf` installed on your machine. But hey, the speed boost is totally worth it!

Fortifying Security: Locking Down Sensitive Data 🔒

Running in bypass or "YOLO" mode gives AI agents broader access to your workspace. Sure, it feels super convenient, but convenience shouldn't come at the cost of your security. If sensitive environment variables like `.env` leak into prompt contexts, they could be exposed in plain text. Yikes! That’s why we run a strict, two-tier security setup.

First, our global rules explicitly forbid the agent from ever reading or spitting out contents from `.env`, AWS credentials, Cloudflare configs, private SSH keys, and token stores. Sensitive files like `.env*`, `.aws/credentials`, `.cloudflared`, `.npmrc`, and `auth.json` are strictly off-limits. If secret values are needed, humans handle them—the AI is strictly limited to non-sensitive identity checks or public data queries.

Here’s the catch: even with global instructions in place, an eager AI agent might still try to take a peek at your `.env` if it thinks it’s relevant. That's why you can't just ask nicely—you need automated guardrails! We enforce this using the deny list in `.claude/settings.json` alongside a `PreToolUse` hook powered by `block-credential-access.py`. This actively scans file access requests and Bash commands, immediately blocking dangerous actions like dumping all env vars, decrypting AWS Secrets Manager, logging Lambda envs, or running `wrangler secret` commands (flagged clearly with a `[CRED-GUARD]` prefix in our logs). Plus, keeping environment files, build artifacts, and cache directories in `.claudeignore` keeps our context clean, lean, and leak-free. 🛡️

Want to set up these "eco-friendly" rules for your codebase yourself? Here's how you can ask your AI agent to keep things neat and green.

Boost Token Efficiency with a Hierarchical Doc Structure

Files like AGENTS.md and CLAUDE.md serve as the ultimate cheat sheet—the default ground rules your AI agent automatically checks before getting to work. Keep this global guide strictly VIP: your tech stack, non-negotiable security protocols, core coding conventions, and quick links to domain-specific docs.

Over time, these global rules love to bloat up (especially when agents eagerly try to add their own rules to the pile). When that happens, it’s time to put your docs on a diet! Prune the fluff and split out anything that doesn't belong. The golden rule? Keep this global file ultra-lean—ideally under 100 lines. Deep-dive details that aren't needed for every single task belong in separate files inside a dedicated docs folder.

Here’s a quick peek at how you can structure it:

This is just a classic example to show the concept in action. If an agent isn't touching billing or logging, why burn valuable tokens dragging those heavy docs into the context? Keep it context-aware, load only what’s needed, and watch your tokens (and wallet) thank you. We follow this exact same hierarchical philosophy across all our docs, including our operational playbooks hosted in separate repos.

Running Custom Skills Like a Pro

Frequent routine tasks—like crafting commits, running code reviews, opening PRs, or handling Git acrobatics—get leveled up into dedicated "Skills." Here at Videostew, our go-to skills look like cm, rv, pull-request, and aws. Skills do way more than save a few keystrokes; they lock down repetitive workflows into reliable SOPs that steadily improve over time.

We build and manage our skills based on two simple rules. First, build them by hand, only when truly necessary. AI loves to over-engineer code and documentation if left unchecked, so we skip the fancy "do-it-all" skills floating around the web. Instead, we start with the simplest bare-bones version tailored to our exact workflow, then refine it organically as edge cases pop up. Second, only automate high-frequency routines. Too many skills bog down session loading times and clutter up recommendations—so if it's a rare task, leave the instructions in a standard doc under docs and keep your toolkit lightning fast.

At the end of the day, a "Skill" is just another document your AI agent reads. Even a standalone `md` file can deliver results just as solid as a built-in skill, as long as you lay out the steps and decision-making logic clearly. The real difference? It’s all about how often you need it, and how easily your agent can find and summon it when magic needs to happen.

Managing Infrastructure, All in One Place

Why drown in complex dashboards when your agent can handle the heavy lifting? We set up our infrastructure so our agent tackles routine checks and repetitive tasks right through command lines: AWS CLI for AWS, Wrangler for Cloudflare, and `gcloud` for Google Cloud. Once the CLI and credentials are ready on the server, you can kiss goodbye to endless back-office clicking. Just tell your agent what you want in plain English, and it figures out the commands and steps. It can even pull real-time traffic and cost metrics to help you forecast bills before any surprises hit.

This is an absolute lifesaver with AWS. While Cloudflare keeps things neat and simple, AWS offers a sprawling galaxy of services that can take hours just to navigate. With your AI co-pilot mapping out how services connect behind the scenes, you can skip the steep learning curves and focus on what truly matters: building great video experiences.

Of course, with great power comes great responsibility! Infrastructure changes directly touch live services. That’s why we always bake a pre-flight check into our guidelines or skills. The agent must verify the goal, exact changes, target resources, and blast radius before pulling the trigger.

Keeping an Eye on Everything: Frictionless Monitoring

Logs are your best friend—whether you're squashing bugs during development or keeping the engine running smoothly for real users in production.

In dev environments, there’s no need to pump logs to costly external platforms; we keep both backend and frontend logs local. On the frontend, we hook into browser `console` methods to stream logs straight to a dedicated logging endpoint, which saves them neatly into files. Backends use their framework’s native logging routes. The coolest part? The AI agent can inspect these logs directly during testing and patch issues on the fly—saving you from ever having to manually copy-paste console errors back and forth again!

Behind the scenes in production, client-side errors take a quick detour through a dedicated ingestion pipeline (powered by AWS Lambda) straight into Amazon CloudWatch. Meanwhile, our backend captures framework-level hiccups using syslog and log agents, streaming everything directly to CloudWatch Logs. To keep our sanity intact, we document exactly which CloudWatch resource holds each log type—and how to query it—in our global engineering guide or dedicated logging docs.

Here’s the real superpower of this setup:

By centralizing error logs in CloudWatch, you can search and debug using a single, unified approach—even when your architecture is split across multiple servers, microservices, and Lambdas. No matter how fragmented logging styles or call stacks might be across services, a single source of truth makes it a breeze for our AI agents (and human devs!) to pinpoint the exact issue and cross-reference the offending code in record time.

Never Letting an Issue Slip Through the Cracks

When pairing with AI agents on development, you quickly realize how fast rabbit holes multiply. You start drilling down into Problem A, only to stumble upon a hidden bug, a prerequisite blocker, or something you've "fixed" that really needs post-release verification. When that happens, we lean heavily on GitHub Issues.

To keep the workflow completely frictionless, we built a custom GitHub Issue skill (just type /issue!). Whenever a side quest pops up mid-development, we summon this skill to spin up a sub-issue on the fly. That way, we can instantly jump right back to crushing original Problem A. These spun-off issues won’t be forgotten, and staying laser-focused on one task keeps the agent’s context lean and pristine—saving precious tokens in the process!

Here’s the catch: many bugs look perfectly fine in isolated code, only acting up when they collide with real user data or live databases. They’re nearly impossible to catch with synthetic dev data, and if they don't trigger a hard error, they can easily vanish into the void. To tackle this, once development wraps up, we create a watch issue—or add an observation plan to the existing issue—with a dedicated monitoring window and check-in date. Using a simple Due: metadata tag on line one alongside custom labels makes tracking effortless. When the time comes, a quick query lets us review the issue against live metrics to confirm whether it's truly resolved or needs another round of polish.

Keep Small Tasks Small, Big Tasks Big

As a scrappy startup (read: our wallets cry easily), we can’t just unleash ultra-high-intelligence AI models for every tiny click. So, we obsess over squeezing maximum efficiency out of every single prompt. When coding or running day-to-day operations, simple repetitive tasks like "check this," "verify that," or "let's tweak this" end up taking a surprisingly massive chunk of the workload. Even before tackling heavy-duty tasks, we often run lightweight probes just to survey the scene. Considering this reality, we set a cost-effective, mid-tier model as our default driver (think Sonnet for Claude, or Luna/Terra for Codex).

To make this work seamlessly, our global instructions teach the default model to recognize heavy lifting and delegate it—calling a high-tier model as a sub-agent when things get hairy (Opus or Fable for Claude, Sol High for Codex). This turns our default model into a smart, cost-conscious router that always picks the right brain for the job without breaking the bank.

Delegating looks like this: the sub-agent scopes the changes and maps out an architectural plan. Once the sub-agent hands over the battle plan, the default model takes over to do the actual coding. It's not completely free of hiccups, of course—delegation can cause a slight bottleneck, and occasionally our default model gets a bit overconfident and botches something it definitely should have passed up the chain.

The fix? Fine-tune your global instructions to auto-delegate whenever high complexity, security, authentication, or DB schema changes pop up—or when a task is taking too long and the scope begins to explode. Establish a crystal-clear division of labor: the Main Agent routes, injects context, writes code, and runs tests, while the Sub-Agent focuses strictly on planning. Escalation thresholds will vary by project, so treat it like tuning an engine: monitor actual runs and adjust the boundaries to fit your workflow.

※ Pro-tip for Claude users: There's an unlisted model called 'opusplan' that lets you decouple your everyday context from dedicated planning mode.

※ And yes—if a task is clearly a monster from second one, we skip the middleman and dive straight in with the heavyweight models.

A Fully Integrated, Living Manual

Refund policies, refund checklists, account rules, operational must-knows... internal playbooks are notoriously massive, and they demand real-time updates every single time a unique edge case pops up. Plus, since the whole team chips in, keeping a spotless version history without nasty merge conflicts is non-negotiable.

Our secret sauce? We manage our entire operational manual in a dedicated Git repository packed with Markdown files. The main README.md acts as the grand central station, mapping out which files govern which domains. By authenticating this repo via standard Git CLI or Git MCP and wiring it as an agent skill (/manual), our AI agents can look up guidelines or push updates on the fly—anytime, anywhere.

※ This hierarchical documentation structure is our go-to pattern across both general guidelines and operational playbooks. It keeps token usage refreshingly lean while maximizing agent efficiency—a win-win we swear by!

A Few More Pro-Tips & Curiosities

--

This covers the broad strokes of how we build and run things agent-first. For anything we missed, we'll dive deeper in upcoming posts, so stay tuned!