Before the Gen AI revolution, we need the API revolution
As a corporate venture capitalist, I have a unique vantage point to watch how Generative AI is transforming the landscape of work. My days are filled with conversations with entrepreneurs, interactions with fellow employees, and hands-on experience with emerging Gen AI tools. This exposure lets me see the impact of AI from various angles, from companies building agents to internal teams adopting tools like Glean.
While we're still in the early stages of Generative AI's integration into office environments, clear use cases are emerging. These applications not only highlight the current state of AI in the workplace but also provide glimpses of what is around the corner.
The Current Landscape
The Basics: Document Creation and Summarization
The most common use of Generative AI in the workplace mirrors its application in personal settings. Search and summarization. Think Perplexity and ChatGPT. In the enterprise, these capabilities are slowly starting to evolve how teams approach memos, reports, presentations, and other forms of text-based communication.
The current capabilities aren’t good enough to present an AI document without a person becoming involved. Instead, AI-generated content serves as a starting point, that is then edited, reshaped, and refined. For instance, when preparing a high-level presentation for executives, AI can quickly generate an initial draft, which team members then substantially revise and enhance.
Gen AI helps get over the “blank paper” issue when creating work artifacts.
A step up: analyzing bulk, unstructured content
Ever increasing context windows make it easy to copy and paste large amounts of content ito an LLM for analysis. This can be used to simplify and extract key points in different business processes. Specific examples I’ve seen are:
Meeting Summaries: Summarize meetings and highlight key points
Customer Service Analysis: Classify the most common types of customer service issues
IT Troubleshooting: Internal help desk tickets and computer log files can be analyzed to spot patterns in technical issues,
This capability extends beyond specific applications to provide a holistic view of organizational operations. Companies like Glean, Akooda, and Fora are leveraging the proliferation of cloud software and APIs to create centralized repositories of organizational data. This enables a new form of "operational intelligence," offering executives and team leaders unprecedented insights into the activities and health of their teams, departments, or entire organizations.
From descriptive to explanatory
As Generative AI models become more sophisticated, their ability to provide explanatory insights (or reasoning) is rapidly improving. This evolution is driven by advancements in underlying models, more refined prompting techniques, and better process management.
A few use cases where AI is providing explanations about behavior:
Algorithm Transparency: Explaining why certain recommendations are made (e.g., why a particular job is being shown to a candidate).
Customer Behavior Analysis: Synthesizing customer service interactions, usage patterns, and churn data to identify potential causes of customer attrition.
Product Development Insights: Analyzing user feedback and feature usage to guide product iterations and improvements.
AI Copilots - The current frontier
Copilots are seeing the most commercial success outside of the foundational model companies themselves. Copilots help workers get more done, more quickly. They have domain specific knowledge and domain specific workflow. AI copilots are now assisting in across the functions within knowledge work: Software Development, Customer Service, Sales, Accounting, HR, marketing, etc.
These AI assistants are capable of producing tangible work artifacts such as code snippets, product requirement documents, and targeted sales messaging. The high-quality, rapidly delivered output can then be refined by in-house teams, outsourced to freelancers for further development, or potentially handled by fully autonomous systems in the future.
Near-Term Predictions
As Generative AI continues to integrate into workplace processes, we can anticipate several significant shifts:
Reduced Reporting Overhead: Automated summarization and analysis will streamline reporting processes, freeing up time for more strategic activities.
Accelerated Information Cycles: Faster data processing and insight generation will lead to more agile decision-making processes.
Rapid Product Development: AI-assisted ideation and prototyping will significantly reduce product development cycles.
Enhanced Scalability: Organizations will be able to handle increased workloads without proportional increases in headcount.
Democratized Software Development: The ability to design custom software solutions will become more accessible, even to those without traditional coding backgrounds.
Unlocking the true value of Gen AI
The flow of summary of documents > describing the situation > diagnosing the problem > generating solutions is laborious. Humans are in the middle reviewing every step, slowing the process down significantly. To get the real value of the technology, some key parts of infrastructure need to develop. And, while other problems need to be solved, e.g., hallucinations, those are well covered by others.
First - the APIs and data ecosystems
Before Gen AI can transform key parts of the customer facing business, there are challenges to overcome. Not just in LLM quality or capability. Access to the right types of organizational data are needed. And, the ability to make sense of (process) and store this data. This means building out a new data ecosystem around Generative AI, inside a company. A quick explanation of data ecosystems and why they matter.
What is a data ecosystem
A data ecosystem is the tools and processes that enable the collection, use, and storage of data within and across organizations. It includes:
Data sources (internal and external) to collect the data
Data wrangling to move and format the data in a useful way
Data processing and analysis tools to make sense of the data
Data storage and management systems to store the data for use
Reporting so everyone can get access to the data
An effective data ecosystem allows for seamless data flow, enabling organizations to derive meaningful insights and power advanced applications like generative AI.
Why the data ecosystem matters - Walmart’s data ecosystem helped it beat out incumbents
To understand the importance of data ecosystems, let's look at a historical parallel from the retail sector, specifically impact of Universal Product Codes (UPC), barcodes, and Enterprise Resource Planning (ERP) systems.
The Birth of UPC and Barcodes
In the early 1970s, the retail industry faced significant challenges in inventory management and pricing. Companies would have to manually check inventory levels, manually tabulate the results, and send off to purchasing teams. The introduction of the Universal Product Code (UPC) and barcode technology in 1974 marked a pivotal moment in retail history.
Barcodes helped organizations automate check outs, providing real-time insights into what products are selling, and what the inventory levels were for the different stores. Companies like Walmart took advantage of these new technologies.
Walmart's Data-enabled Dominance
Walmart, founded in 1962, recognized the potential of these technologies early on and became a pioneer in their adoption and integration. By the 1980s, Walmart installed barcode scanners in all its stores. As mentioned above, this allowed for real-time tracking of sales and inventory levels across locations.
To make sure all of the inventory data was received, Walmart developed its own satellite network in the mid-1980s, connecting all stores to its central computer system. This immediate sharing of sales data led to clearer insights about what was happening across the stores. What to stock, where to stock it, and when to stock it became simpler.
Walmart continued its rise from 1962 to eventually becoming the largest retailer. It did while beating out established incumbents like A&P (The Great Atlantic & Pacific Tea Company), that failed to adopt these emerging technologies.
Step-by-step Walmart was able to take data from new sources (bar code scanners), wrangle and process it with satellites and centralized systems, to lead to better decisions and a competitive advantage. Gen AI will offer something similar for knowledge work, but getting the ecosystem right is the first step.
The Data Ecosystem for Generative AI
For generative AI to reach its full potential in the workplace, a similarly robust and integrated data ecosystem needs to be developed. Here are the key components:
Data Sources
Data sources that were previously not that useful inside an organization, will become useful. Examples are:
Internal communications (emails, chat logs, meeting transcripts)
Employee productivity tools (project management software, time tracking systems)
Customer interaction platforms (CRM systems, support tickets)
External data feeds (market trends, social media, news APIs)
These tools exist and have existed. Storing every text component and then making it accessible to other systems has been less important. Recording of meetings might have been saved so someone that wasn’t there could watch it later. Now, meeting transcripts can be used to help understand what is going on inside an organization: what are key topics of conversation, what are the statuses of sales prospects, what are challenges to the upcoming product launch, etc.
Data Integration (APIs)
To move data between systems, there need to be robust APIs. While there are APIs for most software systems, these APIs may have limitations. Systems of records, generally, do not want to make it easy to move mass amounts of data out. Data gravity is a moat for these systems, creating “anti-gravity” systems are low on priority. Another challenge is often the type of data that can be moved in and out. For example, until recently, Zoom’s API had limited access to meeting transcripts.
Processing and Analysis
To get the most out of the data, there will be need to be Gen AI specific process to “wrangle” the data. Embeddings play a part here, as does RAG. I actually have seen less developed here than other areas. There are copilots and systems outputting work products. There are startups solving storage, but pure processing for data wrangling, it’s been less clear (to me).
Storage
This is no different than previous developments around platforms and data. The benefits of vector databases for storing text embeddings is, perhaps, more useful than before. There is the potential for graph-based databases to help understand relationships between entities.
Combining data sources, APIs, processing, and storage into a coherent system will drive new capabilities for organizations. At this point, there are few Enterprise ready, off the shelf systems. Companies are instead building their own integrated solutions. Klarna stands out as example that is publishing their results, but every enterprise trying to chain together Gen AI outputs is custom assembling their stack.
Implications for the future of enterprises
The wide scale adoption of Generative AI will impact the nature of firms, the roles hired, and how businesses compete.
Some conventional takes I agree with are:
The Rise of Micro-Enterprises: We may see the emergence of "$1B/1 person" companies, where individuals leveraging AI tools can achieve the output and impact traditionally associated with much larger organizations.
Success accruing to incumbents: Large companies, that make investments into generative AI, will continue to take market share from mid-market companies that don’t. Large enterprises will gain market share based on their ability to handle greater complexity in business processes and offerings. Bespoke tasks, and small markets, that were previously uneconomical will become accessible.
I think we will also see the distance between a business and its customers shrink. Sales, customer service, and even marketing serve as mediating layers between the people producing a product and the actual end customers.
Rather than product and engineering teams relying on sales people to relay information or research teams to provide insights, Product & Engineering will get real time, summarize, and prioritized insights about the market, through Gen AI.
Founders, builders, and product managers then often rely on product marketers and others to craft appropriate market messaging to go-to-market. Gen AI can help here as well, building out market messaging for each customer segment.
Ultimately this will enable high scale, high leverage individuals that are able to interact with customers much more quickly. And, ideally, effectively.
The Dawn of a New Era in Knowledge Work
As we stand on the precipice of this Generative AI revolution, it's clear that we're witnessing a paradigm shift in how knowledge work is conducted. From the basic task of document creation to the complex realm of AI copilots, we're seeing the first ripples of what promises to be a tidal wave of change.
The parallels with Walmart's data-driven transformation of retail are striking. Just as barcodes and satellite networks reshaped the competitive landscape of retail, the emerging Generative AI data ecosystem has the potential to redefine the nature of enterprise competition. The winners in this new era will be those who can effectively harness the power of AI to create more agile, responsive, and insight-driven organizations.
However, the true potential of Generative AI remains locked behind the challenges of building robust data ecosystems. The race is on to develop the APIs, processing tools, and storage solutions that will unleash the full capabilities of this technology. As these pieces fall into place, we can expect to see a rapid acceleration in the adoption and impact of AI across all facets of business.
Looking ahead, the implications are profound. The rise of micro-enterprises, the potential for incumbents to extend their lead, and the shortening distance between businesses and their customers all point to a future that is both exciting and uncertain. As the barriers between product creation, marketing, and customer interaction blur, we may see the emergence of a new breed of high-leverage individuals capable of running complex operations with minimal human intervention.
One thing is certain: the workplace of tomorrow will look radically different from today's. As we navigate this transition, the question for every business leader, entrepreneur, and knowledge worker becomes not if, but how they will adapt to and thrive in this AI-augmented future.
The Generative AI revolution is emerging. What products and trends are you seeing?