Timothy-Vinzent commited on
Commit
1bacfe9
·
verified ·
1 Parent(s): 1feb2ff

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +0 -126
app.py CHANGED
@@ -218,132 +218,6 @@ def build_interface():
218
  You can only submit once, so test your system prompt thoroughly before submission!
219
  """)
220
 
221
-
222
-
223
- gr.Markdown("""Classification Task: Document and Clause Level Identification
224
- Challenge Description
225
- Participants must create a system prompt for a language model that classifies user queries about legal documents into two specific categories:"
226
-
227
- 1. Document Level: Determines whether the query refers to a single document or multiple documents.
228
-
229
- 2. Clause Level: Identifies whether the query is focused on:
230
-
231
- - A single clause,
232
-
233
- - Multiple clauses, or
234
-
235
- - General information not constrained to any specific clause.
236
-
237
- The model must return a valid JSON object with the following structure:
238
-
239
- ```json
240
- {"document_level": "single/multiple","clause_level": "single/multiple/general"}
241
- ```
242
-
243
- The goal is to ensure that the model's output is concise, structured, and accurate. This task is designed to evaluate the robustness of the system prompt in handling classification tasks with short, precise outputs.
244
-
245
- <details>
246
- <summary>Click to see example inputs and outputs.</summary>
247
- **Example Inputs and Expected Outputs**
248
-
249
- 1. **User Message Example 1:**
250
-
251
- - *"Please provide the contract for the lease agreement."*
252
-
253
- - **Expected Output:**
254
-
255
-
256
- ```json
257
- {"document_level": "single", "clause_level": "general"}
258
- ```
259
-
260
- 2. **User Message Example 2:**
261
-
262
- - *"I need all clauses related to termination in the employment contract."*
263
-
264
- - **Expected Output:**
265
-
266
-
267
- ```json
268
- {"document_level": "single", "clause_level": "multiple"}
269
- ```
270
-
271
- 3. **User Message Example 3:**
272
-
273
- - *"Can you send me the financial reports and the partnership agreement?"*
274
-
275
- - **Expected Output:**
276
-
277
-
278
- ```json
279
- {"document_level": "multiple", "clause_level": "general"}
280
- ```
281
-
282
- 4. **User Message Example 4:**
283
-
284
- - *"What are the key clauses in the NDA?"*
285
-
286
- - **Expected Output:**
287
-
288
-
289
- ```json
290
- {"document_level": "single", "clause_level": "multiple"}
291
- ```
292
-
293
- 5. **User Message Example 5:**
294
-
295
- - *"Tell me about the company’s financials."*
296
-
297
- - **Expected Output:**
298
-
299
-
300
- ```json
301
- {"document_level": "single", "clause_level": "general"}
302
- ```
303
-
304
- 6. **User Message Example 6:**
305
-
306
- - *"Provide all contracts and their confidentiality clauses."*
307
-
308
- - **Expected Output:**
309
-
310
-
311
- ```json
312
- {"document_level": "multiple", "clause_level": "multiple"}
313
- ```
314
-
315
- 7. **User Message Example 7:**
316
-
317
- - *"Extract the arbitration clause from this service agreement."*
318
-
319
- - **Expected Output:**
320
-
321
-
322
- ```json
323
- {"document_level": "single", "clause_level": "single"}
324
- ```
325
-
326
- </details>
327
-
328
- **Challenge Instructions**
329
- - Design a system prompt that ensures the AI generates outputs like those above when given similar user messages.
330
-
331
- - The system prompt should:
332
-
333
- 1. Specify formatting requirements (e.g., "Output must be a valid JSON object"), not that we are not using constrained decoding or any sort of JSON mode, if not correctly prompted the llm will output plain text.
334
-
335
- 2. Emphasize strict adherence to classification definitions:
336
-
337
- - *Single Document:* Refers to one document.
338
-
339
- - *Multiple Documents:* Refers to more than one document.
340
-
341
- - *Single Clause:* Refers to one specific clause.
342
-
343
- - *Multiple Clauses:* Refers to more than one specific clause.
344
-
345
- - *General Information:* Refers to general content not tied to specific clauses.
346
- """)
347
  gr.Markdown(
348
  "Please enter your details and submit your system prompt below. "
349
  "You can only submit once, I suggest trying to test and build out the system prompt using the same LM being used here elsewhere before submitting."
 
218
  You can only submit once, so test your system prompt thoroughly before submission!
219
  """)
220
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
221
  gr.Markdown(
222
  "Please enter your details and submit your system prompt below. "
223
  "You can only submit once, I suggest trying to test and build out the system prompt using the same LM being used here elsewhere before submitting."