Skip to content

Commit 2918730

Browse files
committed
added ticketing dataset temmplate i.e. ala hello world template for synthetic Data studio
1 parent b866ebb commit 2918730

File tree

2 files changed

+78
-2
lines changed

2 files changed

+78
-2
lines changed

app/core/config.py

Lines changed: 77 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ class UseCase(str, Enum):
1717
LENDING_DATA = "lending_data"
1818
#HOUSING_DATA = "housing_data"
1919
CREDIT_CARD_DATA = "credit_card_data"
20+
TICKETING_DATASET = "ticketing_dataset"
2021

2122
class Technique(str, Enum):
2223
SFT = "sft"
@@ -595,9 +596,46 @@ class UseCaseMetadataEval(BaseModel):
595596
596597
""",
597598

598-
schema=None
599+
schema=None
599600
),
600-
}
601+
602+
UseCase.TICKETING_DATASET: UseCaseMetadata(
603+
name="Ticketing Dataset",
604+
description= "Synthetic dataset for ticketing system ",
605+
topics=["Technical Issues", "Billing Queries", "Payment queries"],
606+
default_examples=[
607+
{
608+
"Prompt": "I have received this message that I owe $300 and I was instructed to pay the bill online. I already paid this amount and I am wondering why I received this message.",
609+
"Completion": "report_payment_issue"
610+
},
611+
{
612+
"Prompt": "I will not be able to attend the presentation and would like to cancel my rsvp.",
613+
"Completion": "cancel_ticket"
614+
},
615+
{
616+
"Prompt": "I am having questions regarding the exact time, location, and requirements of the event and would like to talk to customer service.",
617+
"Completion": "Customer_service"
618+
}
619+
]
620+
,
621+
prompt= """
622+
Generate authentic customer support ticket interactions that have a user query and system response.
623+
For each user query, the system generates a keyword that is used to forward the user to the specific subsystem.
624+
Requirements for user queries:
625+
- Use professional, respectful language
626+
- Follow standard customer service best practices
627+
Each response should be a single id from the following list:
628+
cancel_ticket,customer_service,report_payment_issue
629+
Here are the explanations of the responses:
630+
cancel_ticket means that the customer wants to cancel the ticket.
631+
customer_service means that customer wants to talk to customer service.
632+
report_payment_issue means that the customer is facing payment issues and wants to be forwarded to the billing department to resolve the issue.
633+
634+
""",
635+
schema=None
636+
)
637+
}
638+
601639

602640

603641
USE_CASE_CONFIGS_EVALS = {
@@ -916,6 +954,43 @@ class UseCaseMetadataEval(BaseModel):
916954
917955
918956
Give a score rating 1-10 for the given data. If there are more than 9 points to subtract use 1 as the absolute minimum scoring. List all justification as list.
957+
"""
958+
),
959+
UseCase.TICKETING_DATASET: UseCaseMetadataEval(
960+
name="Ticketing Dataset",
961+
default_examples=[
962+
{
963+
"score": 5,
964+
"justification": """
965+
The query is professionally written, respectful, and follows customer service best practices.
966+
The response 'report_payment_issue' is one of the allowed keywords.
967+
The matching between the query and response is perfect according to the provided definitions.
968+
969+
"""},
970+
{
971+
"score": 3,
972+
"justification": """
973+
The query is professionally written and respectful.
974+
The response 'cancel_ticket' is one of the allowed keywords.
975+
While the response uses a valid keyword, it doesn't match the most appropriate category for the specific query content.
976+
"""
977+
},
978+
979+
],
980+
prompt= """
981+
You are given a user query for a ticketing support system and the system responses which is a keyword that is used to forward the user to the specific subsystem.
982+
Evaluate whether the queries:
983+
- Use professional, respectful language
984+
- Follow standard customer service best practices
985+
Evaluate whether the responses use only one of the the following keywords: cancel_ticket,customer_service,report_payment_issue
986+
Evaluate whether the solutions and responses are correctly matched based on the following definitions:
987+
cancel_ticket means that the customer wants to cancel the ticket.
988+
customer_service means that customer wants to talk to customer service.
989+
report_payment_issue means that the customer is facing payment issues and wants to be forwarded to the billing department to resolve the issue.
990+
Give a score of 1-5 based on the following instructions:
991+
If the responses don’t match the four keywords give always value 1.
992+
Rate the quality of the queries and responses based on the instructions give a rating between 1 to 5.
993+
919994
"""
920995
)
921996
}

app/main.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -773,6 +773,7 @@ async def get_use_cases():
773773
{"id": UseCase.CUSTOM, "name": "Custom"},
774774
{"id": UseCase.LENDING_DATA, "name": "Lending Data"},
775775
{"id": UseCase.CREDIT_CARD_DATA, "name": "Credit Card Data"},
776+
{"id": UseCase.TICKETING_DATASET, "name": "Ticketing Dataset"},
776777
]
777778
}
778779

0 commit comments

Comments
 (0)