Skip to content
Merged
Show file tree
Hide file tree
Changes from 76 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
0c4c3b5
Update
howieleung Oct 22, 2025
d7ea215
agent creation
howieleung Oct 23, 2025
2b731e8
Implement reusable function for conversation management in chat API
howieleung Oct 23, 2025
1fe6bf1
Refactor conversation handling to return Conversation object and save…
howieleung Oct 24, 2025
817d4ab
Refactor get_available_tool to enable index search for Azure AI and s…
howieleung Oct 24, 2025
592fa16
Update error message for missing agent and enhance created_at metadat…
howieleung Oct 24, 2025
e418096
Update Azure AI agent configuration and enhance Application Insights …
howieleung Oct 24, 2025
bd67eb0
Refactor Dockerfile and docker-compose configuration; update requirem…
howieleung Oct 25, 2025
49afea2
Refactor save_created_at function to simplify error handling and remo…
howieleung Oct 25, 2025
0433934
replace whl
howieleung Oct 30, 2025
32806d2
read teaming
howieleung Oct 30, 2025
b932f0f
Update docker-compose configuration, add Azure AI package, and remove…
howieleung Oct 30, 2025
eb8d440
update for instrumentation
howieleung Nov 3, 2025
badabfb
update lib
howieleung Nov 5, 2025
984c189
eval and red teaming
howieleung Nov 5, 2025
c692c30
Refactor agent retrieval methods to use updated API calls and update …
howieleung Nov 8, 2025
fd715ab
Enhance annotation processing in preprocessContent function and strea…
howieleung Nov 9, 2025
756d860
Update README and deployment documentation for Foundry Tools; refacto…
howieleung Nov 11, 2025
88b76d1
Update
howieleung Oct 22, 2025
d6b6c46
agent creation
howieleung Oct 23, 2025
eecccd3
Implement reusable function for conversation management in chat API
howieleung Oct 23, 2025
b314414
Refactor conversation handling to return Conversation object and save…
howieleung Oct 24, 2025
a9af894
Refactor get_available_tool to enable index search for Azure AI and s…
howieleung Oct 24, 2025
aee258a
Update error message for missing agent and enhance created_at metadat…
howieleung Oct 24, 2025
f1dca56
Update Azure AI agent configuration and enhance Application Insights …
howieleung Oct 24, 2025
4bc6b20
Refactor Dockerfile and docker-compose configuration; update requirem…
howieleung Oct 25, 2025
d8010fd
Refactor save_created_at function to simplify error handling and remo…
howieleung Oct 25, 2025
edddc88
replace whl
howieleung Oct 30, 2025
efb588d
read teaming
howieleung Oct 30, 2025
fcb58ee
Update docker-compose configuration, add Azure AI package, and remove…
howieleung Oct 30, 2025
1afd8de
update for instrumentation
howieleung Nov 3, 2025
8f96b99
update lib
howieleung Nov 5, 2025
53fd568
eval and red teaming
howieleung Nov 5, 2025
e06b4ea
Refactor agent retrieval methods to use updated API calls and update …
howieleung Nov 8, 2025
4864ceb
Enhance annotation processing in preprocessContent function and strea…
howieleung Nov 9, 2025
0e59378
Update README and deployment documentation for Foundry Tools; refacto…
howieleung Nov 11, 2025
64d3055
Refactor timestamp handling in AgentPreview component; add formatTime…
howieleung Nov 13, 2025
4dd056b
Refactor agent initialization and output handling; add AZURE_TENANT_I…
howieleung Nov 13, 2025
44eb8dd
Fix bugs
howieleung Nov 13, 2025
d92b67a
improve citation for AI Search
howieleung Nov 13, 2025
29c7263
remove redirect back temporary.
howieleung Nov 13, 2025
879555c
roll back image
howieleung Nov 13, 2025
b127771
Updated readme
howieleung Nov 14, 2025
261675e
update
howieleung Nov 14, 2025
660ef6d
update library
howieleung Nov 14, 2025
513dfe4
update readme and eval
howieleung Nov 15, 2025
fb60542
update role
howieleung Nov 15, 2025
85f3f7e
Merge branch 'howie/agents-v2' of https://github.com/Azure-Samples/ge…
aprilk-ms Nov 15, 2025
39e0200
update
howieleung Nov 15, 2025
a5b04b3
update
howieleung Nov 15, 2025
f9a470c
update cont eval
howieleung Nov 15, 2025
8a58e9c
update tests with .env file path, update md
howieleung Nov 15, 2025
499aaf1
use projects b2
howieleung Nov 15, 2025
68bc697
restore region
howieleung Nov 15, 2025
5b7e607
Merge branch 'howie/agents-v2' of https://github.com/Azure-Samples/ge…
aprilk-ms Nov 16, 2025
d441af6
Apply suggestions from code review
howieleung Nov 16, 2025
78d64de
Changel labels
howieleung Nov 16, 2025
8271156
update eval
howieleung Nov 16, 2025
a2830ca
doc update
howieleung Nov 16, 2025
8ab2df3
update doc
howieleung Nov 16, 2025
4a7734a
Merge branch 'howie/agents-v2' of https://github.com/Azure-Samples/ge…
aprilk-ms Nov 16, 2025
205f00b
updates
aprilk-ms Nov 16, 2025
94f8280
update
aprilk-ms Nov 16, 2025
eb4b683
Use ApiKey
howieleung Nov 16, 2025
6fc7b42
Merge pull request #192 from aprilk-ms/aprilk/eval-updates-1116
howieleung Nov 16, 2025
05a0700
fix ai search
howieleung Nov 16, 2025
5e07941
update searchConnectionId parameter to searchConnectionId_final
howieleung Nov 16, 2025
e16a016
updates
aprilk-ms Nov 17, 2025
39a6bf9
update readme
aprilk-ms Nov 17, 2025
a179c1a
update
aprilk-ms Nov 17, 2025
1fb197e
Update dependencies and improve Dockerfile for frontend build
howieleung Nov 17, 2025
b9ebb90
Merge pull request #193 from aprilk-ms/aprilk/final-touches-agentsv2
howieleung Nov 17, 2025
61d07b8
Resolve comment
howieleung Nov 17, 2025
a5e9abf
improve doc
howieleung Nov 17, 2025
39a7a47
Update deployment guide to reflect changes in Azure AI Foundry quota …
howieleung Nov 17, 2025
f373416
update 4O
howieleung Nov 17, 2025
3eb3389
restucturdd observability docs
changliu2 Nov 18, 2025
0446e0b
removed eval & red-teaming .py script
changliu2 Nov 18, 2025
1e77e47
added back eval & red-teaming .py script
changliu2 Nov 18, 2025
5a06915
relinked README.md to observability.dm
changliu2 Nov 18, 2025
6a9d6de
Merge pull request #195 from changliu2/cliu_updates
howieleung Nov 18, 2025
a0aa721
fix links and better instructions for red teaming
howieleung Nov 18, 2025
6a8090a
eval update
howieleung Nov 18, 2025
9904e1b
update annotation label to use title instead of url
howieleung Nov 18, 2025
29d2709
remove # Version is optional. Defaults to latest version if not spec…
howieleung Nov 18, 2025
ad9ec8e
refactor: remove unnecessary comment about reasoning model in initial…
howieleung Nov 18, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"python.testing.pytestArgs": [
"tests"
],
"python.testing.unittestEnabled": false,
"python.testing.pytestEnabled": true
}
23 changes: 14 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,30 @@
# Getting Started with Agents Using Azure AI Foundry

The agent leverages the Azure AI Agent service and utilizes file search for knowledge retrieval from uploaded files, enabling it to generate responses with citations. The solution also includes built-in monitoring capabilities with tracing to ensure easier troubleshooting and optimized performance.
The agent leverages Foundry Agent Service and utilizes file search for knowledge retrieval from uploaded files, enabling it to generate responses with citations. The solution also includes built-in monitoring capabilities with tracing to ensure easier troubleshooting and optimized performance.

<div style="text-align:center;">

[**SOLUTION OVERVIEW**](#solution-overview) \| [**GETTING STARTED**](#getting-started) \| [**LOCAL DEVELOPMENT**](#local-development) \| [**RESOURCE CLEAN-UP**](#resource-clean-up) \| [**GUIDANCE**](#guidance) \| [**TROUBLESHOOTING**](./docs/troubleshooting.md)

</div>

**Note**: With any AI solutions you create using these templates, you are responsible for assessing all associated risks, and for complying with all applicable laws and safety standards. Learn more in the transparency documents for [Agent Service](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flearn.microsoft.com%2Fen-us%2Fazure%2Fai-foundry%2Fresponsible-ai%2Fagents%2Ftransparency-note&data=05%7C02%7Chowieleung%40microsoft.com%7C42645ec29da244bd920508de2095bcad%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638984024651276233%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=Un4HtoksTeodWPQMQp7zh8BNW6j%2BeIw4mcs6gbS4e6E%3D&reserved=0) and [Agent Framework](https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fagent-framework%2Fblob%2Fmain%2FTRANSPARENCY_FAQ.md&data=05%7C02%7Chowieleung%40microsoft.com%7C42645ec29da244bd920508de2095bcad%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638984024651325701%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=19nfzxn8ZN1qr7Hy77fn%2BgFXD1sc%2BXiuPuUi3H2NNz4%3D&reserved=0).

## Solution Overview

This solution deploys a web-based chat application with an AI agent running in Azure Container App.

The agent leverages the Azure AI Agent service and utilizes Azure AI Search for knowledge retrieval from uploaded files, enabling it to generate responses with citations. The solution also includes built-in monitoring capabilities with tracing to ensure easier troubleshooting and optimized performance.
The agent leverages the Foundry Agent Service and utilizes Azure AI Search for knowledge retrieval from uploaded files, enabling it to generate responses with citations. The solution also includes built-in monitoring capabilities with tracing to ensure easier troubleshooting and optimized performance.

This solution creates an Azure AI Foundry project and Azure AI services. More details about the resources can be found in the [resources](#resources) documentation. There are options to enable logging, tracing, and monitoring.
This solution creates a Microsoft Foundry project and Foundry Tools. More details about the resources can be found in the [resources](#resources) documentation. There are options to enable logging, tracing, and monitoring.

Instructions are provided for deployment through GitHub Codespaces, VS Code Dev Containers, and your local development environment.

### Solution Architecture

![Architecture diagram showing that user input is provided to the Azure Container App, which contains the app code. With user identity and resource access through managed identity, the input is used to form a response. The input and the Azure monitor are able to use the Azure resources deployed in the solution: Application Insights, Azure AI Foundry Project, Azure AI Services, Storage account, Azure Container App, and Log Analytics Workspace.](docs/images/architecture.png)
![Architecture diagram showing that user input is provided to the Azure Container App, which contains the app code. With user identity and resource access through managed identity, the input is used to form a response. The input and the Azure monitor are able to use the Azure resources deployed in the solution: Application Insights, Microsoft Foundry Project, Foundry Tools, Storage account, Azure Container App, and Log Analytics Workspace.](docs/images/architecture.png)

The app code runs in Azure Container App to process the user input and generate a response to the user. It leverages Azure AI projects and Azure AI services, including the model and agent.
The app code runs in an Azure Container App to process user input and generate a response to the user. It leverages Microsoft Foundry projects and Foundry Tools, including the model and agent.

### Key Features

Expand All @@ -38,11 +40,14 @@ Integrated monitoring capabilities, including Azure Monitor and Application Insi
- **[Flexible Deployment Options](./docs/deployment.md)**<br/>
The solution supports deployment through GitHub Codespaces, VS Code Dev Containers, or local environments, providing flexibility for different development workflows.

- **[Continuous Evaluation](./docs/other_features.md#continuous-evaluation)**<br/>
Proactively monitor and assess your agent's performance over time with continuous evaluation that automatically checks real-world interactions to identify potential issues before they impact users.

- **[Agent Evaluation](./docs/other_features.md#agent-evaluation)**<br/>
This solution demonstrates how you can evaluate your agent's performance and quality during local development and incorporate it into monitoring and CI/CD workflow.
This solution demonstrates how you can evaluate your agent's performance and quality through Pytest.

- **[AI Red Teaming Agent](./docs/other_features.md#ai-red-teaming-agent)**<br/>
Facilitates the creation of an AI Red Teaming Agent that can run batch automated scans for safety and security scans on your Agent solution to check your risk posture before deploying it into production.
Facilitates the creation of an AI Red Teaming Agent through Pytest that can run batch automated scans for safety and security on your Agent solution to check your risk posture before deploying it into production.

<br/>

Expand Down Expand Up @@ -111,9 +116,9 @@ The majority of the Azure resources used in this infrastructure are on usage-bas

You can try the [Azure pricing calculator](https://azure.microsoft.com/pricing/calculator) for the resources:

- **Azure AI Foundry**: Free tier. [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/)
- **Microsoft Foundry**: Free tier. [Pricing](https://azure.microsoft.com/pricing/details/ai-studio/)
- **Azure Storage Account**: Standard tier, LRS. Pricing is based on storage and operations. [Pricing](https://azure.microsoft.com/pricing/details/storage/blobs/)
- **Azure AI Services**: S0 tier, defaults to gpt-4o-mini. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/)
- **Foundry Tools**: S0 tier, defaults to gpt-4o-mini. Pricing is based on token count. [Pricing](https://azure.microsoft.com/pricing/details/cognitive-services/)
- **Azure Container App**: Consumption tier with 0.5 CPU, 1GiB memory/storage. Pricing is based on resource allocation, and each month allows for a certain amount of free usage. [Pricing](https://azure.microsoft.com/pricing/details/container-apps/)
- **Log analytics**: Pay-as-you-go tier. Costs based on data ingested. [Pricing](https://azure.microsoft.com/pricing/details/monitor/)
- **Agent Evaluations**: Incurs the cost of your provided model deployment used for local evaluations.
Expand Down
107 changes: 0 additions & 107 deletions airedteaming/ai_redteaming.py

This file was deleted.

4 changes: 2 additions & 2 deletions azure.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

name: azd-get-started-with-ai-agents
metadata:
template: azd-get-started-with-ai-agents@1.0.4
template: azd-get-started-with-ai-agents@2.0.0b1
requiredVersions:
azd: ">=1.14.0"

Expand Down Expand Up @@ -85,4 +85,4 @@ pipeline:
- AZURE_EXISTING_AIPROJECT_ENDPOINT
- AZURE_EXISTING_AGENT_ID
- ENABLE_AZURE_MONITOR_TRACING
- AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED
- OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT
2 changes: 1 addition & 1 deletion docs/deploy_customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ To override any of those resource names, run `azd env set <key> <value>` before

## Customizing model deployments

For more information on the Azure OpenAI models and non-Microsoft models that can be used in your deployment, view the [list of models supported by Azure AI Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support).
For more information on the Azure OpenAI models and non-Microsoft models that can be used in your deployment, view the [list of models supported by Foundry Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support).

To customize the model deployments, you can set the following environment variables:

Expand Down
13 changes: 6 additions & 7 deletions docs/deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,9 +184,9 @@ When you start a deployment, most parameters will have default values. You can c

| **Setting** | **Description** | **Default value** |
|------------|----------------| ------------|
| **Existing Project Resource ID** | Specify an existing project resource ID to be used instead of provisioning new Azure AI Foundry project and Azure AI services. | |
| **Existing Project Resource ID** | Specify an existing project resource ID to be used instead of provisioning new Azure AI Foundry project and Foundry Tools. | |
| **Azure Region** | Select a region with quota which supports your selected model. | |
| **Model** | Choose from the [list of models supported by Azure AI Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support) for your selected region. | gpt-4o-mini |
| **Model** | Choose from the [list of models supported by Foundry Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support) for your selected region. | gpt-4o |
| **Model Format** | Choose from OpenAI or Microsoft, depending on your model. | OpenAI |
| **Model Deployment Capacity** | Configure capacity for your model. | 80k |
| **Embedding Model** | Choose from text-embedding-3-large, text-embedding-3-small, and text-embedding-ada-002. This may only be deployed if Azure AI Search is enabled. | text-embedding-3-small |
Expand Down Expand Up @@ -225,7 +225,7 @@ azd env set ENABLE_AZURE_MONITOR_TRACING true
To enable message contents to be included in the traces, set the following environment variable. Note that the messages may contain personally identifiable information.

```shell
azd env set AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED true
azd env set OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT true
```

You can view the App Insights tracing in Azure AI Foundry. Select your project on the Azure AI Foundry page and then click 'Tracing'.
Expand All @@ -239,9 +239,8 @@ You can view the App Insights tracing in Azure AI Foundry. Select your project o

The default for the model capacity in deployment is 80k tokens for chat model and 50k for embedded model for AI Search. For optimal performance, it is recommended to increase to 100k tokens. You can change the capacity by following the steps in [setting capacity and deployment SKU](deploy_customization.md#customizing-model-deployments).

- Navigate to the home screen of the [Azure AI Foundry Portal](https://ai.azure.com/)
- Select Quota Management buttom at the bottom of the home screen
* In the Quota tab, click the GlobalStandard dropdown and select the model and region you are using for this accelerator to see your available quota. Please note gpt-4o-mini and text-embedding-3-small are used as default.
- Navigate to [Monitor and track your quota usage](https://ai.azure.com/managementCenter/quota)
* In the Quota tab, click the GlobalStandard dropdown and select the model and region you are using for this accelerator to see your available quota. Please note gpt-4o and text-embedding-3-small are used as default.
- Request more quota or delete any unused model deployments as needed.

</details>
Expand Down Expand Up @@ -269,7 +268,7 @@ Once you've opened the project in [Codespaces](#github-codespaces) or in [Dev Co

3. You will be prompted to provide an `azd` environment name (like "azureaiapp"), select a subscription from your Azure account, and select a location which has quota for all the resources. Then, it will provision the resources in your account and deploy the latest code.

- For guidance on selecting a region with quota and model availability, follow the instructions in the [quota recommendations](#quota-recommendations) section and ensure that your model is available in your selected region by checking the [list of models supported by Azure AI Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support)
- For guidance on selecting a region with quota and model availability, follow the instructions in the [quota recommendations](#quota-recommendations) section and ensure that your model is available in your selected region by checking the [list of models supported by Foundry Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support)
- This deployment will take 7-10 minutes to provision the resources in your account and set up the solution with sample data.
- If you get an error or timeout with deployment, changing the location can help, as there may be availability constraints for the resources. You can do this by running `azd down` and deleting the `.azure` folder from your code, and then running `azd up` again and selecting a new region.

Expand Down
Binary file modified docs/images/agent_id_in_foundry_ui.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/agent_monitor.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/enable_cont_eval.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/images/tracing_tab.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading