How we built a first-of-its-kind AI calendar

When Matt Martin and I founded Clockwise in 2016, we set out on a journey to create a solution that would help people better express their needs at work. We came up with a new set of preferences – things like how much time you need in a day for deep work (we call it Focus Time), a shared (and protected!) no-meeting-day for your team, a way to designate the flexibility of a meeting so that productivity can be optimized and enhanced – that would assist teams in making time for the priorities that matter most in their workday.

With these foundations, we engineered an advanced scheduling engine, specifically built to reason about time – the way we think about it, spend it, and optimize it. Notably, this set of sophisticated algorithms was not only powerful enough to understand your individual preferences, but also those of your colleagues. And it had the ability to simultaneously balance the madness of time-zones, working hours, meeting breaks and more.

With these innovations, we brought something new to the workplace and gave voice and value to an improved way to manage your time. Running in the background, the Clockwise scheduling engine powers better schedules, an agent-like experience way ahead of today’s proliferation of agent-based services. Yet still, the experience of finding a time to meet was still messy, but the technology never seemed to be right to more deeply address the pain of scheduling. 

‍

2023: the year AI changed the calendar game

Last year brought some serious advancements with large language models (LLMs), and suddenly, an opening existed for Clockwise to build and deliver something new and different and exciting – for when you need to actively schedule your time. 

Thanks to the powerful natural language processing capabilities of LLMs, for the first time, we saw a path forward to understanding and acting on someone’s calendar intent. Powered by Clockwise’s scheduling engine, which factors in 160 million calendar events on average per day, Prism is the first calendar that combines chat-based assistance with drag-and-click functionality. The result is an intuitive interface that handles complex and manual scheduling better than any human; it’s the most intelligent calendar available.

In our journey to reimagine how we schedule time at work, we learned a few things that I’m excited to dig into today:

• The importance of harnessing proprietary technology to enable highly specific use cases
• The benefits of creating your own Domain Specific Language (DSL) to reduce latency and cost, increase accuracy, and simplify your AI product build
• How the paradigm of the “proposal” helps your users gain confidence in your AI product 
• The power of deep product build that combines natural user behavior with conversational AI

‍

The complicated case of the calendar

“I need 30 mins with Matt (CEO) and Vicky (Head of Product) asap tomorrow for an urgent meeting.”

“Clear out my afternoon and reschedule the meetings I organize, I’ve got a sick kid at home.”

Sometimes you need to meet – you must meet, but you simply can’t find an open time and scheduling feels impossible. When there’s no overlapping whitespace across calendars, people’s schedules get bombed with calendar shrapnel, affecting all the suddenly lesser priority meetings and initiatives. 

Juggling multiple schedules and preferences is a hard problem even for computers to solve and is part of the reason that we, as mere mortals, can sometimes feel so locked in by our calendars and availability when something new comes up. For those lucky enough to have an executive assistant, there’s the possibility of trying to contain the damage. But as humans juggling last-minute priorities, there’s only so much that can be done and so much that can be moved ad-hoc. 

Prism harnesses Clockwise’s proprietary technology to enable highly specific use cases, like scheduling an “impossible” meeting, facilitating last-minute urgent meetings by optimizing on the fly, or rescheduling meetings in bulk when you have a sick kid at home.

All this is made possible because Clockwise uses the powerful natural language processing capabilities of LLMs to translate your calendar request into its proprietary scheduling engine to understand exactly what you want when it comes to your calendar. Married with Clockwise’s ability to analyze vast amounts of data across multiple calendars at once and facilitate the logistics of moving meetings, we’ve built an experience that handles scheduling better than any human. 

The engineering effort to solve seemingly impossible scheduling scenarios was mostly focused on adding capabilities to our internal scheduling engine. It was already adept at handling optimizations based on meeting flexibility across multiple people. The key here was allowing it to be constrained to certain meetings and date ranges. We also made some key product decisions, like on-the-fly optimizations can only consider moving meetings you own and meetings marked as flexible within Clockwise. With those modifications in place, we were able to add capabilities to trigger what we call “fixable conflicts” and “bulk scheduling,” the two example prompts shown above. We also added some additional muscle to the engine to allow it to consider more elaborate optimization strategies, which we’ll explore in a future blog post.

Here’s what that looks like in-product. Notice how the Marketing/Sales Weekly meeting is moved to a spot that works for everyone on Friday, opening up that space for the Project Check-in on Thursday.

Fine tuning AI to translate intent to action 

Prism leverages the natural language processing capabilities of LLMs to understand someone’s calendar request and transform it into a precise, actionable item. Unlike generic LLMs, Clockwise’s system fine-tunes an open-source LLM specifically for calendar management, which ensures reliability when it comes to scheduling. 

Together, AI and LLMs are great at discerning intent via text, but these advancements have no ability to analyze and act on calendar-specific data on their own. LLMs are notoriously bad at math, calendar math especially. 

Our approach is a reverse of how most companies go about building AI experiences, and the way we’ve integrated AI and LLMs makes Clockwise uniquely positioned to transform calendaring. Rather than feeding Clockwise data into AI models for processing, we use AI models to translate someone’s intent into something consumable by our scheduling engine, where all the analysis and calendaring magic happens. 

Clockwise’s approach consumes the intent-level data conveyed via text-based prompts and translates them into an internal domain-specific language (DSL) that we call Clockwise Query Language (CQL), optimized for calendar management. This allows Clockwise to better understand how and when to schedule meetings, making it more useful and accurate.

The translation step to DSL also allowed us to greatly reduce the latency of processing requests within our model. Across all types of language tasks, LLMs tend to perform best at generating programming language output, in this case, CQL. Our scheduling engine then ingests the CQL output from the LLM, interprets it, and produces near-magical scheduling suggestions to get you the time that you need in a way that matches your intent. 

‍

We were also able to improve speed by experimenting with various models that exist in the ecosystem today. GPT4 allowed us to get going quickly, but the large size of the model as well as the need to transmit your request across the wire to OpenAI didn’t allow us to deliver with the speed we were seeking within the experience. We experimented with Gemma 2b, which, as a much smaller model, was much faster but not super adept at handling an ongoing conversation. 

We eventually got the results we were seeking with Llama3 8b, which met the sweet spot of speed and accuracy while being smart enough to understand ongoing context. At the launch of Prism, our average latency for producing CQL from a prompt sits below one second at 0.97s. We are excited to continue driving ever-improving results on performance.

Introducing the Proposal concept

Delivering a resilient “human-in-the-loop” (HITL) interaction model within AI experiences is powerful. From the outset of building Clockwise Prism, we took the stance that HITL would be critical to our success and how we build our AI-native calendar. The idea is simple: you are always in control of your calendar. We don’t want to modify anything on your calendar until you’ve explicitly approved it. Within the Prism interface, we refer to this as a Proposal. 

Its benefits are twofold:

1. Proposals increase comfortability for new users with the system. You can experiment with Prism and preview scheduling options without fear that it’ll destroy your calendar or cancel an important meeting with your manager. Numerous generative AI products suffer from hallucinations, and while our DSL translation step is really great for preventing this from happening within Clockwise, the Proposal step makes sure you are always in charge and only proceed when ready.

2. Proposals allow for modifications before committing a scheduling change. You might want to edit the title, add attendees, or set flexibility settings. Or you may want to explore alternative times, especially when Clockwise Prism is optimizing a “fixable conflict” on the fly. You may just want to visually inspect the before and after placement of a meeting across attendees to help you decide if you actually want to update the time. All of this happens within the Proposal user interface, and once you’re ready hit confirm. Clockwise facilitates all the scheduling logistics and automatically sends out calendar invites so you can move on to the next thing.

‍An AI-native experience with familiar calendar functionality 

To successfully build an LLM-based AI product, you must embrace the old with the new. We are well past the “ChatGPT wrapper” phase of early AI products, and we know this approach doesn’t work to build a durable and valuable experience. Instead, products must embrace a deep product build—the smarts of AI combined with the brick-and-mortar of a point-and-click interface.

At first glance, Prism intentionally looks like a traditional calendar. But as you click around the familiar interface, those interactions result in more intelligent outcomes than what you’d expect with any other cloud calendar. Prism’s smart drag-and-drop means you can move a meeting to the same time as another and Clockwise will automatically offer suggestions to resolve the conflict. Or, click on a meeting to quickly reschedule it.

Notably, Prism offers a first-of-its kind chat experience that lets you talk to your calendar like you would an executive assistant. The chat interface is the high-powered catch-all for more complex, tedious, or manual scheduling needs. Prism’s intelligent AI-driven features are inserted where it benefits the experience; by combining chat functionality with classic calendar behaviors, you can manage your calendar however it’s best for you.

‍

‍

What’s next

The launch of Clockwise Prism is just a moment in time, and we are excited to continue driving improvements and adding functionality to the experience. Hopefully this quick walkthrough of some of the engineering and product challenges sheds light on what it takes to build an AI-native product experience. 

Up next, look out for a weekly series of interviews with the technical team at Clockwise, where we’ll more deeply talk about the challenges we faced building Prism. We’ll explore things like how we solved for the downstream impacts of meeting optimizations, details on creating our domain-specific language, running our own model on AWS, and more. We’ll also follow up with a forward-looking proposal for the future of AI services and agents interacting with one another and a hypothetical way an open multi-agent system could be built to enable AI-native product integrations.

And importantly, while we’re super excited to get our beta experience of Clockwise Prism into your hands, there’s still opportunity and work ahead for us to improve. One example – with this launch we don’t yet support the ability to simply click on an open time slot and schedule time. We know this is needed and are busy at work so this can be delivered in a way that fits our overall approach to augment and inform while we enable functionality. Stay tuned for much more. In the meantime, you can learn more about Prism and all its magic here, or join us on Thursday, September 5 for a co-founder fireside chat where Matt and I discuss the vision behind Prism and how we're tackling scheduling challenges with AI-powered solutions.

It’s an exciting moment to deliver entirely new AI-driven capabilities and outcomes through the augmentation of our existing technology. And there couldn’t be a more perfect name for it than “Prism” – a tool that transforms light into a rainbow of colors – something seemingly magical but is actually just applied science.

‍

‍