Ask Monty
Answer any questions about your feedback like magic
Delve deep into your feedback with “Ask Monty”, an advanced LLM-enabled analysis tool. Pose questions regarding your feedback data, and let Monty analyze, visualize and provide insightful answers.
Key Features
- Ask Monty: Directly ask Monty any query related to the feedback in your workspace.
- Save & Revisit: Save crucial questions for easy future reference. You can find them in your “History”.
- Share Insights: Generate a public link to share the question and its answer with others.
- Set Parameters: Customize the data Monty considers by adjusting parameters in the datasets such as date range, contract value, sources, category, and metadata.
- Citations: Quickly access the feedback used by Monty in generating the response.
- Request Visuals: Ask Monty to plot or visualize your requests to produce shareable visualizations and charts about your query. You can also chat with that visual later in your conversation!
Conversation Navigation
- Example Questions: Get inspired by example questions generated based on recent feedback. These are shown below the chat input on the main Ask Monty page.
- History: View your question history, where you will see the recent conversations you had with Ask Monty.
- Saved Questions: Keep important queries handy by saving them, so you can revisit or re-analyze them whenever you need.
Usage Guide
Ask Monty is Reforge Insight Analytics LLM-enabled chat tool designed to extract deep insights from your feedback data.
The conversation is started by entering an initial query. This query will be used to generate the dataset(s) that will make up the relevant feedback used for the system to reply to each message. The datasets will be constructed via filters extracted from your query, and may include filters on the feedback creation date, category/project labels and metadata values.
You will have the opportunity to edit, add and remove datasets as needed at this step as well.
Once your datasets are confirmed, the Ask Monty engine determines the type of response most applicable to your query. At the time the support intent routes include:
- Text: a query that is best answered via a text response (summarizations, qualitative comparisons, example finding, etc.).
- Charts: a query that is best answered via a chart visualization.
- User Stories: a query that is best answered via a generated user story.
- PRDs: a query that is best answered via a generated draft product requirement document.
- Cohorts: a query that is best answered by creating a cohort of customers all discussing the particular user query.
- Snippet Examples: a query that is best answered by creating and returning a list of relevant snippets to the user query.
Monty will then return a response using the most applicable intent to answer the user query.
After the initial response, the user may ask any number of follow ups or explanations to previous responses (including asking follow ups to chat based responses).
Query Best Practices
- Be Direct: The more direct and targeted your query is, the more accurate the response will be. Avoid vague and short queries like “users mad” or “bugs”. While Monty will do its best to interpret these queries and extract the relevant filters/snippets, the response will be more focused with a more specific query. Better queries could be “Summarize the most common bugs over the past 6 months” or “What are users complaining about in design tagged feedback?”.
- Use Response Type Keywords: Use keywords like “compare”, “show”, “plot”, “visualize”, etc. to guide Monty to the best response type.
- Take Advantage of Filter Extraction: Use text filters like “over the past 6 months”, “from feedback tagged ‘xyz’”, “in app store data”, etc. to specify the data Monty should consider.
Filter Extraction
When a user query is provided, Monty will first extract filters from the query to determine the dataset(s) to use in the response. The filters extracted can include:
- Date Ranges: “over the past 6 months”, “from 2023”, etc.
- Feedback Sources: “from app store reviews”, “from emails”, etc.
- Categories: “bugs about”, “common user questions”, etc.
- Project Areas: “from the spelling bee game project”, etc.
- Tags: “from feedback tagged ‘xyz’”, “from feedback tagged ‘abc’”, etc.
- Metadata: “key themes from feedback with version ‘1.2.3’”, etc.
The extracted filters let us generate the baseline dataset(s) to use in the response. We then apply any secondary text filters to further refine the dataset(s) as needed.
For example the query: “compare the bug reports of the spelling bee game to crossword games over the past 6 months” would extract the filters:
Another example, the query: “show me the sentiment of feedback about dark mode from ios reviews”, where dark mode isn’t a category, tag, or project label, would extract the filter:
with a secondary text filter of dark mode to further refine the dataset.
Chart Support
Ask Monty supports chart based responses when it is determined a figure would be the best way to answer a user query.
While the system is designed to determine the appropriate response type with any user query input, you can ensure a chart based response with keywords similar to “plot”, “show”, “visualize”, etc..
The values that can supply those charts include:
- Feedback source
- Sentiment
- Feedback dates
- Categories
- Project Labels (if enabled)
- Dataset groups from current conversation
- Metadata (for enabled categorical type metadata keys, see general metadata docs)
Dataset Actions
Datset Edits
Dataset filters can be edited, added, or removed at any time following the initial query extraction. When a dataset is editted, any generated responses will be updated to reflect the new dataset.
Dataset Exports
Datasets can be exported at any time to a CSV file for further analysis or sharing. The contacts of the dataset can also be exported to a saved Cohort.
Creating Reports
In chat sessions using one dataset, you can create a report by selecting the “Turn into report” button in top right of the navigation bar. This will generate and save a report using the extracted filters and secondary text filters to create a custom report.