AnswerRocket was founded in 2013 with the vision of creating an intelligent agent that could assist business users with data analysis. Alon Goren, CEO and co-founder, recognized how inefficient it was for business users to wait days or weeks for data analysis and sought to streamline the process for everyone in the enterprise. Our augmented analytics platform was born from the frustration of being unable to obtain quick and accurate answers from data during crucial meetings. By using AI, machine learning, natural language querying, and natural language generation, we were able to make it easier for users to ask questions to get instant insights in plain English.
Fast forward to the launch of ChatGPT in November 2022. The AI landscape has evolved leaps and bounds in just a few short months and presented a unique opportunity to organizations of all industries to consider how they would take advantage of the technology.
We sat down with Alon to get his insights on ChatGPT, large language models, and the evolution of data analysis. He shares how AnswerRocket has layered in ChatGPT with AnswerRocket’s augmented analytics software to create a conversational analytics AI assistant for our customers.
Read the transcript of that interview below.
Question: Why was AnswerRocket started?
Alon Goren: We started AnswerRocket with the idea that anybody should be able to get easy answers from their data and it should be as easy as interacting with a personal assistant. That whole idea came from the frustration of sitting in many meetings where a discussion was had around some critical thing being presented, whether it was a board meeting or a management meeting. A PowerPoint was presented with “here’s the reason why we should do X.”
Inevitably there were follow up questions that couldn’t be answered by the PowerPoint. There were requirements to say can we go out and do analysis? And those would take days or weeks. And that felt very wasteful. It felt like the data is there, why can’t we just go out, ask the question of the data and get back the response? We wanted that experience to be something that was available to everybody in the enterprise.
Question: What does the current AnswerRocket offering include?
Alon Goren: The current AnswerRocket offering is kind of a full pipeline that starts with connecting to data sources and then the end product is some kind of an automatic visualization narrative in response to a user question.
So along the way, the technology we have to build is certainly connecting to a wide range of data sources, including all the major data cloud providers. We built a pipeline that starts with a natural language question that the user is posing, breaks that down to an understanding of how to query the underlying source. Sometimes the analysis requires us to do further things than just querying the database. It requires us to do a forecast or some kind of a machine learning based algorithm to answer the user’s ultimate question. Then the presentation of that answer is in the form of a chart, a narrative, a combination of both those things. The technology to achieve all those are part of the kind of the AnswerRocket modules. Now, when we get into enterprise deployments, which is our core market, there’s lots of surrounding stuff that you have to do so around security and authentication and robustness for enterprise deployment. There’s lots of infrastructure that comes along for the ride. The differentiated modules are kind of at the heart around deep analysis of underlying data and presenting sophisticated answers, but in an easy to read way.
Question: How has the data and analytics space evolved in the last decade?
Alon Goren: AnswerRocket was founded almost ten years ago and since then a lot has happened in the space, a lot has happened with technology in general. I kind of pointed out, I guess, several things. One is the number of data sources that are accessible to enterprise users have grown tremendously. It used to be the case that maybe there was a corporate data warehouse with some critical ERP kind of information in it, maybe basic sales information, but over the years it’s grown to the point where any interaction that happens in the enterprise, most of those interactions are captured digitally, and those interactions can be made into data.
So whether you think about website interactions or HR interaction, customer experience interactions, any of those things usually leave a trail of data behind them. There are more and more digital products or applications that are used by the enterprise, the number of applications for enterprise has probably doubled in those ten years. What we see is just a diversity of kinds of data sources that are accessible, and the need therefore, to accommodate all of those.
The second thing that’s really interesting is the pressure to get answers out of your data in a self service mode has probably increased over time. As the data sets grow, as the kind of questions that could be asked have grown, it puts more and more pressure on the data science team or the data analytics team to field those requests by business users. Because of that pressure, it’s impossible to keep up with that demand.
And so, self-service in theory is the way to solve that problem, where users can ask their own questions and get their own answer. That started with a movement to visualize data with dashboards. Over the years, what’s happened is the proliferation of dashboards has really made it hard for users to find what they’re looking for because they have to understand, “Well, which of the hundred dashboards that I have accessible is the answer actually in?” That evolution of everyone essentially is their own analyst to some degree is a change in the space that technologies have to keep up with. Most significantly, the recent inflection point in large language models has created an opportunity to start dealing with users’ questions in the most natural possible way in terms of language and the response to those questions. I would say the natural language technology stack has really hit that part of the growth curve where everything now appears.
There’s going to be massive disruption and massive changes in the ability to answer users underlying questions.
Question: Why is ChatGPT a revolutionary technology for knowledge workers?
Alon Goren: Technology like ChatGPT is going to have a huge impact on technology, broadly on knowledge workers, probably broadly in many ways for us at AnswerRocket, because we started this journey ten years ago looking for a way to essentially make a solution that feels more like an assistant than a software tool. We’ve been in this mode of trying to understand how we can harness language models and other aspects of natural language processing to achieve that mission. What we see now is that, as is evidenced by the growth of ChatGPT users, that there is a huge appetite for interaction, kind of this natural language level. Right? Before, I would say before the launch of ChatGPT, it was more of an interesting, maybe in academia circles, like the idea of how well is natural language evolving? What problems can it tackle?
Once ChatGPT hit the public web and a million users had access to it within the first week, and something on the order of magnitude of 100 million users have accessed it over time, it has changed the way, I think, the perception of what natural language can achieve. Not just in the sense of “can a machine tease apart what the sentence means, but can a machine carry on a conversation to some productive end?”
Which I think is the biggest kind of revelation with a chat-style interface is that it’s not just about the initial question, it’s about the context of that question phrase and the follow up opportunities to explain what’s in the answer and refine it. So that technology is tremendous. I think it’s going to have a broad impact, not just in analytics, but in any knowledge worker type of tasks where if your interactions to accomplish a job is with a computer, you have to ask the question, well what could that computer be doing for me in a way that doesn’t require me to understand where the buttons and the menu options are in order to achieve whatever I’m trying to do?
Question: How does AnswerRocket use ChatGPT’s large language model?
Alon Goren: We span so many different data sources that a user can connect to and so many different systems that the kinds of questions they can ask are very broad. Our ability to then tackle those questions through the usage of a large language model where we’re not just confined to, “oh, the underlying data that to your question says that the right answer is the number x”, but rather it’s a story that explains what’s going on in the data.
So, for instance, asking a question like, let’s say you work in a consumer goods company and you’re a multinational and you want to know about what’s going on in Southern Europe, how well are we doing versus the competition, that kind of a broad statement implies that there’s an understanding of this competition.
- What’s “my” brands?
- What’s “their” brands?
- How do I measure performance?
- Is it in currency, is it in share, is it in share of volume?
Those are all variables or interesting kinds of KPIs that you can’t answer that question. We use a language model, it lets us back off the idea of saying all the information that’s in the data has to be queried very specifically and narrowly. The final number is X to more of an assessment that says, “oh, we understand in this data set that we have here’s how your business is presented and here’s how the competition presented.”
We’ve gone through that data set and in fact, looked at all things that are of interest to you based on a process where you tell us what you care about. Now we can pull from that information and weave together a story that combines information from any of those kinds of analytics. Not only that, but we actually combine that information with any other information that you have potentially connected us to. For example, if you have PowerPoints or PDFs or other documents or websites that incorporate interesting information that relate to the ultimate problem that you’re trying to solve, those are now accessible. Not just accessible, but summarizable in the same process of looking at your underlying data sources.
You get a much richer story about how things happen and you can have that experience of asking and receiving that story and refining what you’re looking for in a natural language kind of way.
Question: What are the challenges with GPT and other large language models?
Alon Goren: There are many challenges with large language models. They are moving targets though. The kinds of things that we see as challenges today, the techniques by which we solve them will evolve over time.
Kind of a snapshot today would be core issues are:
Hallucinations, which is the idea that the model essentially gives you information that is what you would consider fictional, right? The language models; what they understand is whatever they’ve read and they’ve read a lot of fiction and nonfiction in the course of essentially reading the entire web. The model doesn’t distinguish between those two things per se as far as it’s concerned, you’re asking it to tell a story and it’s going to tell a story and sometimes it’s a fiction writer and sometimes it’s a nonfiction writer just depending on the best resources that it found to answer the question.
In that, our challenge is to make sure because people are asking for factual information from us, right, they want to know what’s going on in the real world, not in some fiction, so we make sure we put the right kind of pre- and post-processing to the natural language model. That means when we ask the question, we provide context to say here is relevant information that you should use in answering your question. In post-processing, meaning we look at the answers that it provides and examine it for truthfulness in terms of does it connect back to the facts. So that is a core challenge. Now, outside of how effective the language model is doing that work, there are things like price and performance that will continue to improve.
There are, let’s say, other technological aspects to it that are a moving target in terms of the kind of information that the model has access to and how to connect to it. For instance, in this recent week actually OpenAI introduced the concept of plugins, which is the idea that you can take a chat experience and extend it, almost like if you think of an app store that lets you download things to your phone or browsers to let you connect plugins. The language model itself serves as a basis for having a conversation across a lot of information that it has. For instance, it doesn’t know real time stock prices, it doesn’t know how to place an order online. Those are things that can be achieved through usage of plugins, meaning that the model has to be taught that if the user is asking to book an appointment somewhere, what tool do I need to achieve that result?
The extension of these models is a very critical area that’s I would say fairly nascent at the moment. We expect that area to grow by a lot in terms of the sophistication and the kinds of things that the models can achieve. AnswerRocket sits in this interesting position where we want to use the model as the basis for the conversation, but we want to augment it both in the sense of providing the tools to answer questions and those tools can appear in the form of a real time interaction with AnswerRocket APIs. Another mechanism is to actually retrieve information and use that as the context. These techniques are called tool augmentation and retrieval augmentation. There are ways of extending what a model can do given that the model is trained on some generic but very broad set of data. The kind of challenges that we face today are engineering challenges by and large of wrestling the existing language models into doing our bidding.
It takes energy to make it suitable for enterprises and our enterprise customers in terms of the end results they get. It doesn’t feel like they’re having a conversation that’s partially fiction, partially nonfiction.
Question: Where will AI and data analysis be in 5 years?
Alon Goren: The pace of the technology and the change in technology for language models and chat experiences is such that there’ll be huge pressure to create very narrow answers or narrow solutions that are really deep for certain fields, right? It’s easy to imagine a world where instead of having one large language model or several large language models that are very broad, those then get operationalized or customized for various use cases. Having an assistant that helps you deal with data analytics could be one of ten assistants that you talk to. They all maybe share some common interface where it’s a team that’s helping as opposed to an individual, but it’s all accessible, let’s say, with the same kind of chat paradigm.
Those deep models, you can imagine each model becoming better and better at serving its users. In our space, we would imagine that if you’re an ecommerce company and you’re trying to do analysis on promotions right. That is probably powered by a bot that’s learned a lot about the ecommerce space and learned a lot about promotional activities and customer behavior, which could be very different from, let’s say, the kind of bot that you’re talking to if you’re trying to do planning for a wedding. Both those scenarios are equally valid in terms of can you have an assistant that helps you do tasks, basically? Anywhere where there is a computer centric task. You have to ask the question, what would a really smart assistant who had access to all the information that it needed to make recommendations me? What could it do for me? And then the possibilities are somewhat endless.
Now, how fast can we realize that vision? It appears that right now, based on the improvements, so if you look at the technology side of it, the capabilities, both in terms of the kinds of information that’s available through a chat bot and the speed at which it operates, those are growing kind of a Moore’s Law or better kinds of numbers. We’re talking about doubling every year or so. That’s because both the hardware and the software are improving in this case, right? Both the algorithms are getting more efficient and the hardware that they’re running on, GPUs, is becoming faster. You get this kind of effect that multiplies those two improvements and that unleashes at the moment. When we look at large language models increasing the size of the parameters that they use, increasing the amount of data that they see, increasing the amount of time they get to train on that data, all those have not been tapped out.
All those seem to continue to add capabilities. Those emergent capabilities create a future where you say, okay, what questions shouldn’t it be able to answer, right? If it’s given access to all the information it needs, what are the emergent things that we will find? Because it was a total surprise that suddenly language models can write poems in any number of styles, the creative side of doing tasks was not the thing that AI was supposed to automate. It was supposed to automate routine things, not things that we consider creative tasks. It’s been very surprising to see that actually, as it learns more and more about language and content, that language or the world through text, but the capabilities have increased tremendously. If you put it back down to the concept of five years from now, I feel like this kind of conversation will probably be one or more assistants on this call, and they’ll participate in some ways that help you sharpen your answer, help you better understand the content.
It feels like there’ll be a world where whatever you write to communicate, an assistant will help you. Whoever’s reading might say, well, just give me a summary across all of the information I got. It feels like we’ll have a system of both the receiving and the sending side of the conversation in various ways.
Question: What unique value does Max deliver?
Alon Goren: The way we approach building Max and what we think we could achieve is unique and very valuable.
Probably first and foremost revolves around the idea of getting deep within the customer base that we choose to serve. We’re not trying to necessarily go across all industries and all use cases. We’re trying to be much more targeted because we believe that by being targeted you can get much deeper. You could better build a deep understanding of what users want and how to deliver it. Especially in the case of having conversational type interactions.
It’s challenging or currently impossible to teach a bot to know everything about all kinds of questions. If we focus on something really interesting, like we spend a lot of time with consumer goods companies, we spend time with the finance companies and healthcare more broadly, in each of those areas, there are needs to understand their domain, understand their particular, not just the vocabulary, but what drives the business.
- How do they measure performance of a business?
- How do they set up the objectives for those businesses?
- How do they go about their work, of planning out what to do?
- Where are the opportunities, where are the threats?
The unique thing that we can bring to the table is working closely with those customers to ensure that the domain that we build, the knowledge that we build into the system to work alongside them, is second to none.
That’s in contrast to probably the broadest solutions that are out there, that are designed more to be platforms to serve all use cases, right? Where they have to be agnostic about the kind of data and the kind of questions that they answer, which I think will work great for a lot of broad use cases, but won’t be the best choice for those companies who have a need for deeper analysis and the desire to automate more of the work that they’re doing.
In Conclusion
Looking ahead, the advancements in language models and chat experiences hold tremendous potential for the field of AI and data analysis. The future may see the emergence of specialized AI assistants customized for specific domains, capable of providing deep and tailored insights. These assistants, powered by advanced large language models, could transform various industries by offering efficient and personalized assistance. As technology continues to improve, with hardware and software enhancements complementing each other, the possibilities for AI and data analysis are expanding rapidly. With ongoing developments, the vision of having smart assistants with access to vast amounts of information and the ability to provide valuable recommendations is within reach. The trajectory of progress suggests that the limitations of what these assistants can achieve will continue to be pushed, unlocking new and unforeseen possibilities in the near future.
To learn more about the Data Superpowers within your reach, watch our YouTube playlist here.
