Agentic Workflows for Unlocking User Engagement Insights

What Are Agentic Workflows?

Agentic workflows are systems or processes where we define the goal and the role and provide some context around both of them, and the AI agents determine the steps to fulfill the goal. They differ slightly from AI automation, where we define the goal and role and provide the steps to accomplish the goal. 

Advantages of Agentic Workflows

There are a lot of advantages to using agentic workflows. Here are a few that are showcased in our article today.

• Autonomy. Agents are designed to operate independently, empowered to make decisions and adapt to changing conditions without needing supervision from a human.
• Modularity. Agents bring modularity to the product or use case, which will be showcased in our agentic workflow below, where the product or use case can be broken down into smaller, well-defined components.
• Tool integration. Agents provide effective integration of tools or APIs to make interacting with external processes or systems easier.

Agentic Workflows to Gather User Engagement Insights

Let's say we have a web application for which we have gathered some user data, such as event count per day, session count per day, and the average session time per day. Suppose we would like to gather some insights from this data, such as the story the data tells us, and also come out with action items on how the insights can be used to make changes to the web application to improve upon certain things. In that case, it is a very tedious task solely for the reason that the data has to be viewed cumulatively to arrive at some insights and should also be extrapolated to come out with action items based on the insights.

This is where agentic workflows come in handy; we provide goals to the AI agent by providing the data, and the agent takes the necessary steps to achieve the goals. There are a lot of frameworks to help us orchestrate agentic workflows, such as LangGraph, Auto-Gen, and CrewAI. I have used CrewAI for this particular use case.

User Engagement Analyzer

The user engagement analyzer is an agentic workflow with two agents, a "User Engagement Data Analyst" and a "Report Compiler" agent. The agentic workflow is illustrated in the diagram below.

The User Engagement Data Analyst Agent uses the "Analyze User Engagement Data" task to analyze the user engagement data.

For the purpose of this demonstration, I have three fabricated spreadsheets with user engagement information: event count per day, average session time per day, and session count per day for a few days. 

The idea is to provide this information to the User Engagement Data Analyst Agent to analyze the information and hand over that information to the report compiler agent. The Report Compiler Agent receives the output from the previous agent and compiles a report that provides insights and actionable items in a straightforward manner. 

We'll bring these agents together via CrewAI.

Agent Definition

Now, let us take a look at how the agents are defined in detail via CrewAI.

        analyzer = Agent(
            role='User Engagement Data Analyst',
            goal='Analyze user engagement  data to identify trends, patterns, and key insights',
            backstory="""You are an experienced user engagement data analyst with expertise in 
            analyzing data to extract valuable insights. 
            You are the best at reading and gaining insights about average session time on a given day, 
            event count on a particular day session count on a particular day looking at these over a period of time """,
            tools=[self.excel_read_tool],
            llm=self.llm,
            verbose=True
        )

        compiler = Agent(
            role='Report Compiler',
            goal='Compile analysis into structured, actionable reports with clear recommendations',
            backstory="""You are a skilled report writer who excels at organizing 
            complex information into clear, actionable insights. You focus on 
            creating reports that drive business value.""",
            llm=self.llm,
            verbose=True
        )

Every agent is provided with a role (which role should the agent perform), goal (what is the end result expected from the agent) and backstory (context around the role and goal provided to the agent), tool (which tool should the agent use to perform the task provided) and the LLM to be used for achieving the goal.  

As you can see, we have defined both the agents used in our current case. 

Task Definition

Now, let us define the tasks that will be performed by the agents.

     # Task 1: Analyze User Engagement Data
        analysis_task = Task(
            description=f"""
            Analyze the following Excel files one by one:
            - {self.file_paths[0]} (Event Count per Day)
            - {self.file_paths[1]} (Average Session Time per Day)
            - {self.file_paths[2]} (Session Count per Day)
            
            For each file:
            1. Use the "Read Excel File" tool to read it
            2. Review the JSON data returned
            3. Extract key insights and metrics
            
            Provide specific insights about:
            - Key trends and patterns
            - Important metrics and their changes
            - Any notable findings
            
            Be specific and quantitative in your analysis.
            """,
            expected_output="""
            A detailed analysis including:
            - Key metrics and trends from each file
            - Important patterns and insights
            - Notable findings and observations
            """,
            agent=analyzer
        )

        # Task 2: Compile Report
        compilation_task = Task(
            description="""
            Create a structured report including:
            1. Executive Summary
            2. Key Findings and Metrics
            3. Detailed Analysis
            4. Areas for Improvement
            5. Recommendations
            
            Make all insights specific and actionable.
            Include concrete metrics where possible.
            """,
            expected_output="""
            A complete report with:
            - Clear executive summary
            - Specific findings and metrics
            - Actionable recommendations
            - Areas for improvement
            """,
            agent=compiler
        )

The important point to note in the above code is that at the end of the task definition, we define the agent with which the task is associated. The analyze user engagement data task is associated with the analyzer agent, and the compile report task is associated with the report compiler agent.

Crew Definition

Now, let us take a look at how everything is wrapped together using a Crew.

crew = Crew(
            agents=[analyzer, compiler],
            tasks=tasks,
            process=Process.sequential,
            verbose=True
        )

The code above is a very simplistic implementation of a Crew where the two agents, analyzer and compiler, are bound together. As you can see, the process is set to execute sequentially, which means the analyzer works first, and the compiler uses the output of the analyzer.

Output

Let us now take a look at the output of the User Engagement Analyzer Crew.

For ease of access, I stored the output from the crew in a .md file.

# User Engagement Analysis Report

*Generated on: 2025-01-15*

---

Executive Summary:
Our analysis of the event count, session time, and session count data indicates significant day-to-day variability in all three measures. However, no discernible pattern or trend has been identified across the period. The highest event count (848) and session count (49) were both observed on 2024-10-22 while the longest average session time of 20.17 minutes was recorded on 2024-10-18.

Key Findings and Metrics:
1. Event Count: Fluctuates between 111 and 848, with no clear increasing or decreasing trend.
2. Session Time: Ranges from 3.9 minutes to 20.17 minutes, with considerable day-to-day variability.
3. Session Count: Varies from 6 to 49 sessions per day, without any discernible pattern.

Detailed Analysis:
The data shows a lack of consistency in the event count, session time, and session count. The highest and lowest measures for each metric do not correlate with each other, suggesting the absence of an overarching trend. This unpredictability could indicate that external factors are affecting user engagement on a daily basis.

Areas for Improvement:
1. Data Collection: To better understand the observed variability, consider collecting additional data, such as user demographics or behavioural patterns.
2. Analysis: Apply advanced statistical techniques to potentially uncover hidden trends or correlations within the data.
3. User Engagement: Explore strategies to increase the consistency of user engagement, such as personalized notifications or rewards for regular usage.

Recommendations:
1. Enhance data collection methods to gather more comprehensive data, providing a more holistic view of user engagement.
2. Utilize advanced analytical methods to further investigate the underlying causes of the significant day-to-day variabilities.
3. Implement engagement strategies to promote consistent user interaction and potentially stabilize the event count, session time, and session count.

As you can see, with the help of this agentic workflow, we could easily get an executive summary of the user engagement data analysis. The report contains key findings and metrics, a detailed analysis, areas for improvement, and recommendations. 

The power of the agentic workflow is that it makes a lot of research and analysis work quickly and easily. It is like an assistant working for you who could shave out a lot of research and analysis effort for you.

Next Steps

Now that we have looked at how user engagement data can be autonomously read, analyzed and also plotted for action items, we will now look at what additions we can bring to this agentic workflow to make it more powerful.

1. Introduce data collector agent. In our current agentic workflow, we supply event count, session count, and session duration data to the agent, but instead, we could have another agent whose role is to look at all the user analytic data that is available and bring in the details that are necessary for the analyzer agent become more powerful in the analysis.
2. Data validation agent. We could have a data validation agent validate the quality of the collected data, completeness and consistency. It could also identify anomalies and outliers in the data.
3. Multiple report formats. The output report can be exported in various other formats to help with easy access to the report and encourage usage of the report.
4. Enhanced tool usage. We could also use tools that enable the forecasting or prediction of data and help us do competitive analysis through enhanced web search. 

Now that you understand how agentic workflows can be implemented easily, try to use them to analyze user engagement data and share your insights.

Resources

• CrewAI's website