You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/user_guide/large_language_model/autogen_integration.rst
+78Lines changed: 78 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -104,3 +104,81 @@ Following is an example LLM config for the OCI Generative AI service:
104
104
},
105
105
}
106
106
107
+
Logging And Reporting
108
+
=====================
109
+
110
+
ADS offers enhanced utilities integrating with OCI to log data for debugging and analysis:
111
+
* The ``SessionLogger`` saves events to a log file and generates report to for you to profile and debug the application.
112
+
* The ``MetricLogger`` sends the metrics to OCI monitoring service, allowing you to build dashboards to gain more insights about the application usage.
113
+
114
+
Session Logger and Report
115
+
-------------------------
116
+
117
+
To use the session logger, you need to specify a local directory or an OCI object storage location for saving the log files.
118
+
A unique session ID will be generated for each session. Each session will be logged into one file.
119
+
Optionally, you can specify the ``report_dir`` to generate a report at the end of each session.
120
+
If you are using an object storage location as ``report_dir``, you can also have a pre-authenticated link generated automatically for viewing and sharing the report.
121
+
122
+
.. code-block:: python3
123
+
124
+
from ads.llm.autogen.v02.loggers import SessionLogger
125
+
126
+
session_logger = SessionLogger(
127
+
# log_dir can be local dir or OCI object storage location in the form of oci://bucket@namespace/prefix
128
+
log_dir="<AUTOGEN_LOG_DIR>",
129
+
# Location for saving the report. Can be local path or object storage location.
130
+
report_dir="<AUTOGEN_REPORT_DIR>",
131
+
# Specify session ID if you would like to resume a previous session or use your own session ID.
132
+
session_id=session_id,
133
+
# Set report_par_uri to True when using object storage to auto-generate PAR link.
134
+
report_par_uri=True,
135
+
)
136
+
137
+
# You may get the auto-generated session id once the logger is initialized
138
+
print(session_logger.session_id)
139
+
140
+
# It is recommended to run your application with the context manager.
141
+
with session_logger:
142
+
# Create and run your AutoGen application
143
+
...
144
+
145
+
# Access the log file path
146
+
print(session_logger.log_file)
147
+
148
+
# Report file path or pre-authenticated link
149
+
print(session_logger.report)
150
+
151
+
The session report provides a comprehensive overview of the timeline, invocations, chat interactions, and logs in HTML format. It effectively visualizes the application's flow, facilitating efficient debugging and analysis.
152
+
153
+
.. figure:: figures/autogen_report.png
154
+
:width:800
155
+
156
+
Metric Logger
157
+
-------------
158
+
The agent metric logger emits agent metrics to `OCI Monitoring <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Concepts/monitoringoverview.htm>`_,
159
+
allowing you to integrate AutoGen application with OCI monitoring service to `build queries <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Tasks/buildingqueries.htm>`_ and `dashboards <https://docs.oracle.com/en-us/iaas/Content/Dashboards/Concepts/dashboardsoverview.htm>`_, as well as `managing alarms <https://docs.oracle.com/en-us/iaas/Content/Monitoring/Tasks/managingalarms.htm>`_.
160
+
161
+
.. code-block:: python3
162
+
163
+
from ads.llm.autogen.v02 import runtime_logging
164
+
from ads.llm.autogen.v02.loggers import MetricLogger
165
+
166
+
monitoring_logger = AgentMonitoring(
167
+
# Metric namespace required by OCI monitoring.
168
+
namespace="<MY_NAMESPACE>",
169
+
# Optional application name, which will be a metric dimension if specified.
170
+
app_name="order_support",
171
+
# Compartment OCID for posting the metric
172
+
compartment_id="<COMPARTMENT_OCID>",
173
+
# Optional session ID to be saved as a metric dimension.
174
+
session_id="<SESSION_ID>"
175
+
# Whether to log agent name as a metric dimension.
176
+
log_agent_name=False,
177
+
# Whether to log tool name as a metric dimension.
178
+
log_model_name=False,
179
+
# Whether to log model name as a metric dimension.
0 commit comments