Skip to content

Commit d95e4ea

Browse files
committed
Update docs.
1 parent 389193d commit d95e4ea

File tree

2 files changed

+78
-0
lines changed

2 files changed

+78
-0
lines changed

docs/source/user_guide/large_language_model/autogen_integration.rst

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,3 +104,81 @@ Following is an example LLM config for the OCI Generative AI service:
104104
},
105105
}
106106
107+
Logging And Reporting
108+
=====================
109+
110+
ADS offers enhanced utilities integrating with OCI to log data for debugging and analysis:
111+
* The ``SessionLogger`` saves events to a log file and generates report to for you to profile and debug the application.
112+
* The ``MetricLogger`` sends the metrics to OCI monitoring service, allowing you to build dashboards to gain more insights about the application usage.
113+
114+
Session Logger and Report
115+
-------------------------
116+
117+
To use the session logger, you need to specify a local directory or an OCI object storage location for saving the log files.
118+
A unique session ID will be generated for each session. Each session will be logged into one file.
119+
Optionally, you can specify the ``report_dir`` to generate a report at the end of each session.
120+
If you are using an object storage location as ``report_dir``, you can also have a pre-authenticated link generated automatically for viewing and sharing the report.
121+
122+
.. code-block:: python3
123+
124+
from ads.llm.autogen.v02.loggers import SessionLogger
125+
126+
session_logger = SessionLogger(
127+
# log_dir can be local dir or OCI object storage location in the form of oci://bucket@namespace/prefix
128+
log_dir="<AUTOGEN_LOG_DIR>",
129+
# Location for saving the report. Can be local path or object storage location.
130+
report_dir="<AUTOGEN_REPORT_DIR>",
131+
# Specify session ID if you would like to resume a previous session or use your own session ID.
132+
session_id=session_id,
133+
# Set report_par_uri to True when using object storage to auto-generate PAR link.
134+
report_par_uri=True,
135+
)
136+
137+
# You may get the auto-generated session id once the logger is initialized
138+
print(session_logger.session_id)
139+
140+
# It is recommended to run your application with the context manager.
141+
with session_logger:
142+
# Create and run your AutoGen application
143+
...
144+
145+
# Access the log file path
146+
print(session_logger.log_file)
147+
148+
# Report file path or pre-authenticated link
149+
print(session_logger.report)
150+
151+
The session report provides a comprehensive overview of the timeline, invocations, chat interactions, and logs in HTML format. It effectively visualizes the application's flow, facilitating efficient debugging and analysis.
152+
153+
.. figure:: figures/autogen_report.png
154+
:width: 800
155+
156+
Metric Logger
157+
-------------
158+
The agent metric logger emits agent metrics to `OCI Monitoring <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Concepts/monitoringoverview.htm>`_,
159+
allowing you to integrate AutoGen application with OCI monitoring service to `build queries <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Tasks/buildingqueries.htm>`_ and `dashboards <https://docs.oracle.com/en-us/iaas/Content/Dashboards/Concepts/dashboardsoverview.htm>`_, as well as `managing alarms <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Tasks/managingalarms.htm>`_.
160+
161+
.. code-block:: python3
162+
163+
from ads.llm.autogen.v02 import runtime_logging
164+
from ads.llm.autogen.v02.loggers import MetricLogger
165+
166+
monitoring_logger = AgentMonitoring(
167+
# Metric namespace required by OCI monitoring.
168+
namespace="<MY_NAMESPACE>",
169+
# Optional application name, which will be a metric dimension if specified.
170+
app_name="order_support",
171+
# Compartment OCID for posting the metric
172+
compartment_id="<COMPARTMENT_OCID>",
173+
# Optional session ID to be saved as a metric dimension.
174+
session_id="<SESSION_ID>"
175+
# Whether to log agent name as a metric dimension.
176+
log_agent_name=False,
177+
# Whether to log tool name as a metric dimension.
178+
log_model_name=False,
179+
# Whether to log model name as a metric dimension.
180+
log_tool_name=False,
181+
)
182+
# Start logging metrics
183+
runtime_logging.start(monitoring_logger)
184+
286 KB
Loading

0 commit comments

Comments
 (0)