MilanM commited on
Commit
a84c3b0
·
verified ·
1 Parent(s): 5d4e71d

Update app.py

Browse files
Files changed (1) hide show
  1. app.py +7 -215
app.py CHANGED
@@ -1,16 +1,7 @@
1
  import streamlit as st
2
  from io import BytesIO
3
- import requests
4
  import time
5
- import re
6
-
7
- # ReportLab imports
8
- from reportlab.lib import colors
9
- from reportlab.lib.pagesizes import A4
10
- from reportlab.lib.styles import getSampleStyleSheet, ParagraphStyle
11
- from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer
12
- from reportlab.lib.units import mm
13
-
14
 
15
  # Set page config
16
  st.set_page_config(
@@ -47,7 +38,6 @@ def check_password():
47
  if not check_password():
48
  st.stop()
49
 
50
-
51
  # Initialize session state
52
  if 'current_page' not in st.session_state:
53
  st.session_state.current_page = 0
@@ -62,18 +52,10 @@ if 'answers' not in st.session_state:
62
  'trust_focus': '',
63
  'trust_source': '',
64
  'trust_warranted': '',
65
- 'trust_conclusion': ''
 
66
  }
67
 
68
- def format_text(text):
69
- # Make text before colon bold
70
- text = re.sub(r'(^|[\n\r])([^:\n\r]+):', r'\1##\2**:', text)
71
-
72
- # Make text in parentheses italic
73
- text = re.sub(r'\(([^)]+)\)', r'*(\1)*', text)
74
-
75
- return text
76
-
77
  # Define the content for each page
78
  pages = [
79
  {
@@ -90,154 +72,7 @@ pages = [
90
  Foundation Model: Evaluate what part Llama2 as a model in the chatbot may play in the risks.
91
  """
92
  },
93
- {
94
- 'title': "Identifying Main Stakeholder Groups",
95
- 'content': """
96
- We need to discover the most pressing governance barriers for our client.
97
- Specify up to three main stakeholder groups whose trust in the solution must be established for long-term success.
98
- """,
99
- 'input_key': 'stakeholder_groups',
100
- 'input_type': 'text_area',
101
- 'example': """
102
- Gen.ai chatbot solution for law firm example:
103
- Group 1 - End-users (attorneys/legal counsellors)
104
- Group 2 - IT Department (providing the tooling)
105
- Group 3 - End-clients (the plaintiffs or defendants)
106
- """
107
- },
108
- {
109
- 'title': "What is Opacity?",
110
- 'content': """
111
- Through this process we will look out for factors known as "Opacity", they represent limiters that prevent us from making informed decisions, knowing why things happen or making sure we can take accountability for issues that come up.
112
- They are not just technological problems, but societal ones as well.
113
-
114
- There are three main types we will focus on (most common and among the most dangerous):
115
- Intentional Concealment / Technological Literacy / Cognitive Mismatch
116
- """
117
- },
118
- {
119
- 'title': "Intentional Concealment",
120
- 'content': """
121
- Intentional Concealment is a form of opacity that occurs when organizations prevent or reduce access to information
122
- so that they can maintain their trade secrets or competitive advantages. It's always present in some shape or form with
123
- proprietary technologies, maybe they won't declare where they get their data, maybe they won't declare the techniques
124
- they use to train their models, etc.
125
- """,
126
- 'input_key': 'intentional_concealment',
127
- 'input_type': 'text_area',
128
- 'example': """
129
- Gen.ai chatbot solution for law firm example:
130
- Concealment 1: They don't let people audit the training data.
131
- Concealment 2: We can't deploy the model ourselves, only consume it from them via an endpoint.
132
- Transparency 1: They published an academic paper on how the model was developed.
133
- """
134
- },
135
- {
136
- 'title': "Technological Literacy",
137
- 'content': """
138
- Technological Literacy (often termed as technological illiteracy in literature) is a form
139
- of opacity that most impacts the general population because understanding these technologies is not a common or easily accessible
140
- skill. Often the thing that a user experiences is very different than what the technology actually does. It's similar
141
- to how a person may know how to use a smartphone, but not to code an app for one, or to engineer a phone themselves.
142
- """,
143
- 'input_key': 'technological_literacy',
144
- 'input_type': 'text_area',
145
- 'example': """
146
- Gen.ai chatbot solution for law firm example:
147
- Group 1 (Attorneys) - Yes, Yes : They should be able to evaluate the output if it seems abnormal due to their domain expertise, but may not understand how the solution works in the background.
148
- Group 2 (IT Department) - Yes, Yes : They have used foundation models for other tasks before to create other solutions.
149
- Group 3 (Plaintiffs/Defendants) - Yes, No : They aren't familiar with legal precedents, that's why they hired our client to help them. They need to trust their expertise but less so the solution.
150
- """
151
- },
152
- {
153
- 'title': "Cognitive Mismatch",
154
- 'content': """
155
- Cognitive Mismatch is the most difficult form of opacity, it's something we can only moderately protect ourselves from
156
- but never get rid of. It happens when we are dealing with systems, technologies or just scales that are way beyond our
157
- human point of reference. In AI systems it most often happens when we need to know exactly how a system came up
158
- with a decision. This is a problem inherent to both Predictive and Generative AI, it is the cause of the AI Black Box that you may have heard of before.
159
- """,
160
- 'input_key': 'cognitive_mismatch',
161
- 'input_type': 'text_area',
162
- 'example': """
163
- Gen.ai chatbot solution for law firm example:
164
- Group 1 - Yes: The chatbot invents a new type of precedent by hallucinating, but it turns out that it is a real thing that we never expected or noticed!?
165
- Group 2 - Yes: The government wants us to provide an explanation of why a model said what it said to a client and ruined their life, how do I do that when it has 70 billion parameters!?
166
- Group 3 - Yes: In the meeting the chatbot starts to tell the client that they will suffer 40 years of jail time for causing the death of the Dalai Lama because they didn't pay the taxes.
167
- """
168
- },
169
- {
170
- 'title': "Intrinsic and Extrinsic Trust",
171
- 'content': """
172
- Intrinsic trust comes from us, we believe we understand something enough or have enough
173
- experience from it. So we trust ourselves and the way we engage with the technology.
174
- Extrinsic trust comes from outside, when we rely on someone/something's reputation, the
175
- recommendations of experts, reports about their credibility and so forth. But it is also much more
176
- frail than intrinsic trust.
177
-
178
- Which form of trust do we want to focus on, and what is the source of that trust? Can it be further strengthened?
179
- """,
180
- 'input_key': 'trust_focus',
181
- 'input_type': 'combined',
182
- 'options': ['Intrinsic trust', 'Extrinsic trust'],
183
- 'example': """
184
- Gen.ai chatbot solution for law firm example:
185
- We choose to rely on Intrinsic Trust in this case. Because our client's customers come to them because they trust them, while the IT department trusts that they can handle this solution and we trust the expertise of our lawyers to handle any anomalies. But we can also provide the IT department with tools to help them maintain their intrinsic trust in their work by monitoring it.
186
-
187
- Source of Trust: The intrinsic trust comes from the lawyers' expertise in their field and the IT department's familiarity with similar systems. We can further strengthen this by providing regular training sessions on the AI system and its limitations, as well as implementing a robust feedback mechanism for continuous improvement.
188
- """
189
- },
190
- {
191
- 'title': "Warranted or Unwarranted Trust",
192
- 'content': """
193
- Warranted trust meaning that we have enough arguments to say that we firmly believe that
194
- we can trust ourselves to handle the usecase alongside our client, their customers and the
195
- solution itself.
196
-
197
- Unwarranted trust meaning that we still choose to trust our approach and case even though we
198
- don't have enough arguments and may be doing it for irrational reasons such as "it feels like it can do
199
- it good" or "the results are the only thing that matters, even if we can't make sure the tech is
200
- sound."
201
-
202
- Based on your answers and thought process above, determine if our trust in this case is warranted or unwarranted.
203
- Then, create a short conclusion on why the trust is or isn't warranted based on the answers from the previous sections.
204
- """,
205
- 'input_key': 'trust_warranted',
206
- 'input_type': 'combined',
207
- 'options': ['Trust is warranted', 'Trust is unwarranted'],
208
- 'example': """
209
- Gen.ai chatbot solution for law firm example:
210
- Conclusion: We believe that trust in our solution is Warranted when we consider the subject matter expertise of our clients internal legal and IT experts, as well as our recommendation of using some watsonx.governance tooling to help bolster the IT department's intrinsic trust in the solution and reduce cognitive mismatch opacity.
211
-
212
- This trust is warranted because:
213
- 1. The legal experts (end-users) have the domain knowledge to critically evaluate the chatbot's outputs.
214
- 2. The IT department has experience with similar AI systems and will be equipped with additional monitoring tools.
215
- 3. We've identified potential opacity issues and have plans to address them, such as using explainable AI techniques.
216
- 4. The solution will be continuously monitored and improved based on feedback from all stakeholder groups.
217
- """
218
- },
219
- {
220
- 'title': "Improving Trust",
221
- 'content': """
222
- Based on your evaluation, how can we improve upon this trust?
223
- Consider the different types of opacity discussed earlier and the specific needs of each stakeholder group.
224
- Provide concrete suggestions for strengthening trust in the AI solution.
225
- """,
226
- 'input_key': 'trust_improvement',
227
- 'input_type': 'text_area',
228
- 'example': """
229
- To improve trust in our example gen.ai chatbot for the law firm:
230
-
231
- 1. Transparency: Develop a clear, non-technical explanation of how the AI works for all stakeholders.
232
- 2. Training: Provide regular training sessions for attorneys on how to effectively use and critically evaluate the chatbot's outputs.
233
- 3. Feedback Loop: Implement a robust feedback mechanism where users can flag incorrect or concerning outputs, feeding into continuous improvement.
234
- 4. Explainability: Integrate explainable AI techniques to provide rationale for the chatbot's recommendations, addressing cognitive mismatch.
235
- 5. Auditing: Establish regular audits of the system's performance and decision-making processes, sharing results with stakeholders.
236
- 6. Ethical Guidelines: Develop and prominently display clear ethical guidelines for the AI's use in legal contexts.
237
- 7. Collaboration: Foster ongoing collaboration between legal experts and IT teams to bridge the technological literacy gap.
238
- 8. Gradual Rollout: Implement the solution in phases, allowing time for trust to build and for refinements based on real-world use.
239
- """
240
- },
241
  {
242
  'title': "Generate Evaluation Report",
243
  'content': "You have completed the AI Trust and Opacity Evaluation. Click the button below to generate and download your PDF report.",
@@ -245,46 +80,6 @@ pages = [
245
  }
246
  ]
247
 
248
- def generate_pdf():
249
- buffer = BytesIO()
250
- doc = SimpleDocTemplate(buffer, pagesize=A4, rightMargin=20*mm, leftMargin=20*mm, topMargin=20*mm, bottomMargin=20*mm)
251
-
252
- # Define custom colors
253
- dark_purple = colors.Color(0.2, 0, 0.4) # Very dark purple for titles
254
- light_purple = colors.Color(0.6, 0.4, 0.8) # Lighter purple for answers
255
- medium_gray = colors.Color(0.5, 0.5, 0.5) # Medium gray for descriptions
256
-
257
- styles = getSampleStyleSheet()
258
- styles.add(ParagraphStyle(name='SectionTitle', fontName='Helvetica', fontSize=16, spaceAfter=6, textColor=dark_purple))
259
- styles.add(ParagraphStyle(name='Description', fontName='Helvetica', fontSize=12, spaceAfter=6, textColor=medium_gray))
260
- styles.add(ParagraphStyle(name='Answer', fontName='Helvetica', fontSize=12, spaceAfter=12, textColor=light_purple))
261
- styles.add(ParagraphStyle(name='MainTitle', fontName='Helvetica-Bold', fontSize=16, spaceAfter=24, textColor=dark_purple))
262
-
263
- story = [Paragraph("AI Trust and Opacity Evaluation", styles['MainTitle'])]
264
-
265
-
266
- for page in pages[:-1]: # Skip the last page
267
- if 'input_key' in page and page['input_key'] is not None:
268
- story.append(Paragraph(page['title'], styles['SectionTitle']))
269
- story.append(Paragraph(page['content'], styles['Description']))
270
-
271
- answer = st.session_state.answers.get(page['input_key'], "")
272
- if isinstance(answer, list):
273
- answer = ', '.join(answer)
274
-
275
- if page.get('input_type') == 'combined':
276
- option = st.session_state.answers.get(page['input_key'], "")
277
- conclusion = st.session_state.answers.get(f"{page['input_key']}_conclusion", "")
278
- answer = f"Option selected: {option}\n\nConclusion: {conclusion}"
279
-
280
- story.append(Paragraph(f"<b>Answer:</b> {answer}", styles['Answer']))
281
- story.append(Spacer(1, 12))
282
-
283
- doc.build(story)
284
- buffer.seek(0)
285
- return buffer
286
-
287
-
288
  # Streamlit app
289
  st.title("AI Trust and Opacity Evaluation")
290
 
@@ -296,7 +91,7 @@ with col1:
296
  st.session_state.current_page -= 1
297
  st.rerun()
298
  with col3:
299
- if st.session_state.current_page < len(pages) - 1: # Changed condition to allow "Next" on the second-to-last page
300
  if st.button("Next", use_container_width=True):
301
  st.session_state.current_page += 1
302
  st.rerun()
@@ -342,7 +137,7 @@ if 'input_key' in current_page and current_page['input_key'] is not None:
342
  # Generate PDF button (only on the last page)
343
  if st.session_state.current_page == len(pages) - 1:
344
  if st.button("Generate and Download PDF", use_container_width=True):
345
- pdf = generate_pdf()
346
  st.download_button(
347
  label="Download PDF",
348
  data=pdf,
@@ -352,7 +147,4 @@ if st.session_state.current_page == len(pages) - 1:
352
  )
353
 
354
  # Display progress
355
- st.progress((st.session_state.current_page + 1) / len(pages))
356
-
357
- # st.divider()
358
- # st.info("Developed by Milan Mrdenovic © IBM Norway 2024")
 
1
  import streamlit as st
2
  from io import BytesIO
 
3
  import time
4
+ from report_gen import generate_pdf
 
 
 
 
 
 
 
 
5
 
6
  # Set page config
7
  st.set_page_config(
 
38
  if not check_password():
39
  st.stop()
40
 
 
41
  # Initialize session state
42
  if 'current_page' not in st.session_state:
43
  st.session_state.current_page = 0
 
52
  'trust_focus': '',
53
  'trust_source': '',
54
  'trust_warranted': '',
55
+ 'trust_conclusion': '',
56
+ 'trust_improvement': ''
57
  }
58
 
 
 
 
 
 
 
 
 
 
59
  # Define the content for each page
60
  pages = [
61
  {
 
72
  Foundation Model: Evaluate what part Llama2 as a model in the chatbot may play in the risks.
73
  """
74
  },
75
+ # ... (include all other pages here)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
76
  {
77
  'title': "Generate Evaluation Report",
78
  'content': "You have completed the AI Trust and Opacity Evaluation. Click the button below to generate and download your PDF report.",
 
80
  }
81
  ]
82
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
  # Streamlit app
84
  st.title("AI Trust and Opacity Evaluation")
85
 
 
91
  st.session_state.current_page -= 1
92
  st.rerun()
93
  with col3:
94
+ if st.session_state.current_page < len(pages) - 1:
95
  if st.button("Next", use_container_width=True):
96
  st.session_state.current_page += 1
97
  st.rerun()
 
137
  # Generate PDF button (only on the last page)
138
  if st.session_state.current_page == len(pages) - 1:
139
  if st.button("Generate and Download PDF", use_container_width=True):
140
+ pdf = generate_pdf(pages, st.session_state.answers)
141
  st.download_button(
142
  label="Download PDF",
143
  data=pdf,
 
147
  )
148
 
149
  # Display progress
150
+ st.progress((st.session_state.current_page + 1) / len(pages))