Video summary
The video demonstrates a test-driven approach to fixing bugs within an agentic workflow using the Debugger Agent in Visual Studio Copilot, illustrated through a real-world example involving the open-source project Friend. The process begins by submitting a GitHub bug link describing an intermittent "index out of range" exception; since no concrete reproduction steps are provided, the agent acknowledges the lack of information but proceeds to analyze the code starting from the exception's throw site. By examining the stack trace and existing test infrastructure, the agent formulates hypotheses about the root cause, identifying that a method returns true without verifying if an archetype is empty, which leads to the error when iterating over an empty collection.
To validate these hypotheses, the agent leverages its ability to generate new tests dynamically. It creates a specific test case designed to reproduce the exact exception by setting up entities with matching archetypes and then deleting some to leave an empty one for iteration. After building the solution and running this newly generated test, the agent confirms that it fails with the expected stack trace. The workflow then moves into a deep debugging phase where breakpoints are placed in the code to inspect runtime states and evaluate expressions, ultimately confirming the hypothesis that skipping empty archetypes is necessary to prevent the crash.
Once the root cause is verified, the agent proposes a fix involving a do-while loop to ensure empty archetypes are skipped before iteration. After reviewing the proposed change, the user approves the fix, prompting the agent to run the generated test again to ensure it now passes without throwing an exception. The agent then offers to run a broader suite of related tests to check for regressions; upon approval, it executes 73 tests, finding that only one pre-existing failure remains, thereby confirming that the new fix resolves the issue without introducing new problems.
The session concludes with the agent summarizing the entire debugging journey, from fetching bug details and formulating hypotheses to generating failing tests, inspecting runtime states, applying a code fix, and verifying stability across a larger test suite. This end-to-end experience highlights how the Debugger Agent can autonomously navigate complex debugging scenarios by combining code analysis with test-driven development principles. The video encourages viewers to try this agentic workflow on their own codebases to streamline bug resolution and invites feedback on how different models handle these iterative verification loops in various contexts.
Read the full video transcript
Let's take a look at the debugger
agent's bug fix workflow using the
test-driven path today. Uh let's use
friend, which is an entity component uh
open source project on GitHub. I'll use
this bug unhandled index out of range
exception. It has a small stack trace
and it says it happens intermittently.
Let me copy this bug URL. And uh here in
Visual Studio Copilot chat, let me
switch to the debugger agent in the
drop-down.
I'll flick the size breaker prompt where
it says paste a GitHub or an ADO bug
link or describe the bug. I'll replace
that with my bug link. You can also use
a bug description in here or an ADO bug
link. So debug bug link, let's submit
this. And what this does essentially, it
fetches the work item details from
GitHub, like the title, bug description,
anything else that's in there. So uh it
says I cannot confidently generate repro
steps. The issue is intermittent and uh
there's not enough information. It's
asking me basically if I have any
further information that could help
reproduce this issue before it proceeds.
Uh I don't, so I'll just say no,
continue with whatever is available.
Let's see what it did so far. So it
started analyzing code based on this
exception and starting at the throw
site. Now I see the issue. Let me look
at the move next and here it has a bug.
Um so it it has a theory on what's going
on here. When advancing to the next
archetype, it returns true without
checking if that archetype is empty,
having zero entities. So that's its
primary hypothesis. It has a few other
alternate hypothesis as well. And it did
look at the existing test infrastructure
in this code base, uh found some
existing tests. There's already one
failing test in this code base. It ran
this test to see if it matches. This
test fails, but with a different issue.
So, the exception stack trace didn't
match. So, then it looked through other
existing tests. So, now uh it said, "Let
me write a test that creates this
entities with matching arch types and
deletes some
to leave an empty one, and then iterate
the query." So, it generated that test
now, and it's building the solution
before querying the test to find this
newly generated test. It found this test
now, and it's running through it to make
sure it fails. Uh looks like it failed
with that same exception stack trace.
So, now it's debugging through it to
confirm the hypothesis. So, it placed
some break points in the code. It hit
the break point. Now, looks like it's
evaluating expressions and inspecting
the runtime state. And this break point
continues to the next break point.
Okay, it says confirmed. Confirmed its
hypothesis now. So, root cause
confirmed, and it has a proposed fix.
Doesn't skip empty arch types. Um
sounds reasonable. Do while loop to skip
empty arch types. Okay, that fix looks
reasonable to me, so I'll say yes, apply
the fix.
Now, it's running the test that it
generated previously to make sure it
passes.
Does this fix fully resolve the issue?
Looks like uh when it ran the test, it
passed, and uh we didn't see that
exception. And that fix looks good to
me, so I'll say yes. Different models
behave differently, and then again, uh
the same model behaves differently with
the different runs sometimes.
So, uh in the case of this bug, it ran
the test and made sure it passes. In
some cases, it'll decide to do another
loop with debugging the test again
uh with the fix and looking at the break
points, inspecting the runtime state,
just to do a thorough check to verify
that the fix resolves the issue. In this
case, it's confident enough running the
test, it passed. So, it's moving on to
summarizing the root cause and the fix.
Now, it's asking me if I want to run
related test check for regressions. So,
I could say yes or no in this case. To
be on the safe side, let me say yes to
run all the related test make sure I
didn't introduce any regressions with
this fix. That was quick. Run a bunch of
tests. 73
tests
and one test failed. This is the one
failing test that is already failing
before our change. Fair enough. No
regressions introduced and the fix is
complete. All right. So, to summarize
what we saw today, we given a a GitHub
bug and asked the debugger agent to
debug this. It fetched in all the bug
details from GitHub. Found that there's
no concrete repro steps provided. So, it
looked through the code using the stack
trace and then it formulated some
hypothesis. It looked at the existing
test infrastructure, the existing tests.
Couldn't find anything that reproduces
the same issue. So, it generated a new
test that would fail with the exact same
exception stack trace. It debugged
through the test with breakpoints,
inspected the runtime state when the
breakpoints hit, and it confirmed the
hypothesis. Once it confirmed the
hypothesis, it proposed a fix. The fix
looked reasonable, so we asked it to
apply the fix. And then it verified the
fix by running the test again. The test
passed. No exceptions this time. So, it
summarized the issue, the root cause,
and the fix for us and gave us an option
to run broader set of tests. I said yes
to that, so it ran all these tests and
made sure it didn't introduce any
regressions. That's the whole end to end
debugging experience with Copilot using
the bug fix agentic flow in the debugger
agent. This is using the test-driven
path. So, please try this out with your
own code base and the bugs that you
have, and please give us feedback. Thank
you.