Industries have begun tapping into the potential of transforming meeting transcripts into actionable notes. But with the evolution of prompt-engineering and retrieval-augmented generation architectures, there’s a world of untapped potential. Here’s a glimpse into the capabilities we’ve developed and what lies ahead on our roadmap (R), for real examples click links:
- Meeting Analysis: Pinpoint roadblocks, bottlenecks, and provide actionable recommendations.
- User Persona: Recognize demographics, behaviors, and motivations to guide feature design and quantify insights.
- User Journeys(R): Detail the entire user experience, spotlighting touchpoints, friction zones, and delightful moments.
- User Stories: A staple in agile methodologies, user stories encapsulate specific user needs in a concise format.
- Coding Recommendations(R): With a foundational understanding from user stories and requirements, offer code insights on optimization, redundancy removal, or potential dependencies.
Simulated Presentation before UAT
- Bugs & Issues(R): Capture details from user feedback and testing sessions, and receive code recommendations to resolve.
- User Documentation: Clear and concise user documentation for new product releases that outlines new features, their benefits and how to use them.
- User Guides(R): Introduction to product, getting started, step by step guides for key features, common issues and solutions, FAQs, glossary, and links to additional support and resources.
Vision: Product Team Accelerator
ProdSage boosts product team efficiency in two primary areas: client engagement and development operations. Our AI Copilot enhances engagement by analyzing conversational data, linking insights to a knowledge graph, and paving the way for content and code automation.
In the fast-paced business environment, transmitting intricate business and technical knowledge, especially to new team members, poses a significant challenge. Traditionally, senior team members bear this burden, juggling both hands-on tasks and mentoring.
ProdSage ensures every team member is equipped with deep knowledge of the client and product, regardless of tenure. This knowledge grows and evolves as projects progress.
Technology: Architecture and Learning
Our MVP harnesses Google Cloud, Drive, and Colab in tandem with OpenAI’s API. It processes meeting transcripts to deliver user-requested content. Early iterations, using models like GPT-3.5-Turbo-16K and GPT-4, showed promise but lacked consistency. Key learnings from our journey:
Data Pre-Processing:
- Entity Extraction – Enhanced prompts with Microsoft’s Guidance library infused with entity details improved context steering.
- Transcript Editing – Meeting transcriptions often have disjointed statements. Parsing and combining these statements not only refines the text but also preps it for embedding.
- Embedding Database – By creating Name : Statement documents and ingesting into a free open-source embedding database like ChromaDB you’re able to create more sophisticated prompt-engineering with RAG.
Prompt-Engineering (Guidance Tips & Tricks):
Tradeoffs between quality and cost are ubiquitous, especially when architecting prompts for conversational AI. While there are countless methods to structure prompts, I’ll delve into a select few, sharing my insights.
One-Shot: System Message, User Message Context Blob, Assistant Message with Settings
This approach is likely the most cost-efficient. By not simulating a back-and-forth dialogue between a fictional user and an assistant, you avoid the redundancy of repeating tokens in successive API calls. In essence, this method streamlines the process, optimizing for cost without compromising quality.
summarize_program = guidance("""
{{#system~}}
You are a product manager's assistant that generates detailed summary of conversational transcripts between a product manager and their end user.
{{~/system}}
{{#user~}}
Here is a conversational transcript between me: {{userName}} the Senior Product Manager of {{productDescription}} Conversational Transcript: {{text_content}}
Please generate a detailed summary of this conversation, {{objective}}.
{{~/user}}
{{#assistant~}}
{{gen 'transcriptSummary' temperature=0.1 max_tokens=5000}}
{{~/assistant}}
""")Conversation Approach: Iterative Progressive Conversations for Enhanced Context Steering
While this method can be costlier due to the accumulation of past tokens with each subsequent generation, it significantly bolsters the quality of context. By progressively building on the conversation, it provides a more nuanced and accurate context steering.
meeting_summarize_program = guidance("""
{{#system~}}
You will be provided with a document delimited by triple exclamation points and an objective. You are an expert product manager and your task is to fulfill the objective using only the provided
document and user context. If there is not sufficient information to answer this question then simply write: "Insufficient information."
{{~/system}}
{{#user~}}
!!! {{text_content}} !!!
Objective: Summarize the meeting notes in 2 paragraphs.
{{~/user}}
{{#assistant~}}
{{gen 'meetingSummary' temperature=0.1 max_tokens=1000 presence_penalty=0 frequency_penalty=.2}}
{{~/assistant}}
{{#user~}}
Objective: Write a markdown list of the speakers and each of their key points.
{{~/user}}
{{#assistant~}}
{{gen 'meetingKeyPoints' temperature=0.1 max_tokens=1000 presence_penalty=0 frequency_penalty=.2}}
{{~/assistant}}
{{#user~}}
Objective: Finally, list the next steps or action items suggested by the speakers, if any.
{{~/user}}
{{#assistant~}}
{{gen 'meetingNextSteps' temperature=0.1 max_tokens=1000 presence_penalty=0 frequency_penalty=.2}}
{{~/assistant}}
""")Final Thoughts
In the initial stages of my prompt-engineering journey, I’ve shared just two approaches tailored to this use case. But rest assured, this is merely the beginning. Stay tuned for deeper dives and more advanced techniques in upcoming posts. To ensure you don’t miss out, subscribe to our newsletter for email notifications on our latest insights. I’m eager to hear from you – let me know in the comments what you’d like to explore further or what you believe requires less emphasis. My aim is to continually share valuable insights on generative AI with you all.
