Moly

Moly is a Makepad app to interact with local and remote LLMs. You can know more about it on GitHub.

Moly Kit is a Rust crate with widgets and utilities for Makepad to streamline the development of artificial intelligence applications. It is used by Moly and can be used in your own Makepad apps.

The following chapters are dedicated to Moly Kit, the crate.

Basics

This section covers everything you need to know to integrate Moly Kit into your Makepad application.

After completing the Quickstart, you will have a working standalone chat in your app. This is made possible by the batteries-included Chat widget. Further chapters will guide you on customizing it to your specific needs.

Quickstart

Prerequisites

This guide assumes you are familiar with Makepad and you have a bare-bones app ready to start integrating Moly Kit while following this guide.

Installation

Add Moly Kit to your Cargo.toml dependencies:

moly-kit = { git = "https://github.com/moxin-org/moly.git", features = ["full"], branch = "main" }

Tip: Change branch = "main" to (for example) tag = "v0.2.1" if you want to stay on a stable version.

If you are targeting native (non-web) platforms, you will also need to add tokio to your app. Even if you don't use it directly, Moly Kit will.

cargo add tokio -F full

#[tokio::main]
async fn main() {
    your_amazing_app::app::app_main()
}

Note: tokio is not needed if you are only targeting web platforms. More details about targeting web will be covered in the Web support guide.

Register widgets

As with any Makepad app, we need to register the widgets we want to use in the live_register of your app before any widget that uses Moly Kit.

impl LiveRegister for App {
    fn live_register(cx: &mut Cx) {
        makepad_widgets::live_design(cx);
        
        // Add this line
        moly_kit::live_design(cx);

        crate::your_amazing_widgets::live_design(cx);
    }
}

DSL

Import the batteries-included Chat widget into your own widget and place it somewhere.

live_design! {
    use link::theme::*;
    use link::widgets::*;

    // Add this line
    use moly_kit::widgets::chat::Chat;
    

    pub YourAmazingWidget = {{YourAmazingWidget}} {
        // And this line
        chat = <Chat> {}
    }
}

Rust-side configuration

The Chat widget as it is will not work. We need to configure some one-time stuff from the Rust side.

The Chat widget pulls information about available bots from a synchronous interface called a BotContext. We don't need to understand how it works, but we need to create and pass one to Chat.

A BotContext can be directly created from a BotClient, which is an asynchronous interface to interact with (mostly remote) bot providers like OpenAI, Ollama, OpenRouter, Moly Server, MoFa, etc.

Once again, we don't need to understand how a BotClient works (unless you need to implement your own) as Moly Kit already comes with some built-in ones. We can simply use OpenAIClient to interact with any OpenAI-compatible remote API.

We should ensure this configuration code runs once and before the Chat widget is used by Makepad, so a good place to write it is in Makepad's after_new_from_doc lifecycle hook. The practical tl;dr of all this theory would be simply the following:

use moly_kit::*;

impl LiveHook for YourAmazingWidget {
    fn after_new_from_doc(&mut self, _cx: &mut Cx) {
        let mut client = OpenAIClient::new("https://api.openai.com/v1".into());
        client.set_key("<YOUR_KEY>".into());

        let context = BotContext::from(client);

        let mut chat = self.chat(id!(chat));
        chat.write().bot_context = context;
    }
}

Note: Moly Kit doesn't duplicate methods from Chat into Makepad's autogenerated ChatRef but provides read() and write() helpers to access the inner widget.

Use multiple providers

Prerequisites

This guide assumes you already read the Quickstart.

Mixing clients

As seen before, a BotContext is created from one (and only one) BotClient.

If we want out app to be able to use multiple clients configured in different ways at the same time, we will need to compose them into one.

Fortunally, Moly Kit comes with a built-in client called MultiClient, which does exactly that. MultiClient can take several "sub-clients" but acts as a single one to BotContext, routing requests to them accordengly.

Going back to our configuration from the Quickstart, we can update it to work with several clients at the same time:

use moly_kit::*;

impl LiveHook for YourAmazingWidget {
    fn after_new_from_doc(&mut self, _cx: &mut Cx) {
        let client = {
          let mut client = MultiClient::new();

          let mut openai = OpenAIClient::new("https://api.openai.com/v1".into());
          openai.set_key("<YOUR_KEY>".into());
          client.add_client(openai);

          let mut openrouter = OpenAIClient::new("https://openrouter.ai/api/v1".into());
          openrouter.set_key("<YOUR_KEY>".into());
          client.add_client(openrouter);

          let ollama = OpenAIClient::new("http://localhost:11434/v1".into());
          client.add_client(ollama);

          client
        };

        let context = BotContext::from(client);

        let mut chat = self.chat(id!(chat));
        chat.write().bot_context = context;
    }
}

Integrate and customize behavior

Prerequisites

This guide assumes you have already read the Quickstart.

Introduction

As we saw before, we only need to give a BotContext to Chat, and then it just works. The Chat widget is designed to work on its own after initial configuration, without you needing to do anything else.

However, when integrating it into your own complex apps, you will eventually need to take control of specific things. Chat allows you to do just that in a way tailored to your needs through a "hooking" system.

Since this is an important concept to understand, let's start by explaining the theory.

Hooks and tasks

When Chat wants to perform a relevant interaction, like sending a message, updating texts, copying to the clipboard, etc., it will NOT do that without giving you the chance to take control.

To do so, those relevant interactions are defined as "tasks", and they are grouped and emitted to a callback that runs just before Chat executes the action.

That callback is what we define as a "hook". A hook not only gets notified of what is about to happen, but also gets mutable access to the group of tasks, meaning it can modify them or abort them as needed.

So, in other words, "tasks" are "units of work" that are about to be performed, but we can "tamper" with them.

The set_hook_before method

The set_hook_before method can be used during the configuration of the Chat widget to set a closure that will run just before Chat performs any relevant action.

It will receive a &mut Vec<ChatTask> as its first parameter, which is the representation of the actions that will be performed as part of a group.

Chat uses the information from inside a ChatTask to perform the real action, so modifying their data will impact how the action is executed.

Additionally, as this is exposed as a simple mutable vector, you can clear() it to essentially prevent Chat from doing anything with them.

This is basically an "abort" mechanism, similar to a web browser's preventDefault() method in Event.

Of course, you can do anything you want with this vector, like injecting more tasks, swapping them, etc.

Enough theory. Let's see a practical example. We will modify the setup code from the Quickstart to configure a hook:

use moly_kit::*;

impl LiveHook for YourAmazingWidget {
    fn after_new_from_doc(&mut self, _cx: &mut Cx) {
        // ... the previous setup code from quickstart ...

        chat.write().set_hook_before(|group, chat, cx| {
          // If we set this to true, the group of tasks will be cleared and
          // the default behavior will not happen.
          let mut abort = false;

          // We don't care about grouping right now so let's just deal with them
          // individually.
          for task in group.iter_mut() {
            // Let's log to the console when sending a message.
            if let ChatTask::Send = task {
              println!("A message is being sent!");
            }

            // Let's ensure a message always gets updated with uppercase text.
            if let ChatTask::UpdateMessage(_index, message) = task {
              message.content.text = message.content.text.to_uppercase();
            }

            // Let's prevent the default behavior of copying a message to the clipboard
            // and handle it ourselves to add a watermark.
            if let ChatTask::CopyMessage(index) = task {
              abort = true;

              let messages = chat.messages_ref();
              let text = messages.read().messages[*index].content.text.clone();
              let text = format!("You copied the following text: {}", text);

              cx.copy_to_clipboard(&text);
            }
          }

          if abort {
            group.clear();
          }
        });
    }
}

Okay, that was a very comprehensive example that is worth a hundred words.

Why are tasks grouped?

Tasks are grouped because not all UI interactions map to a single unit of work.

For example, the "Save and regenerate" button will trigger two grouped tasks: SetMessages to override the message history, and then Send to send the history as it is.

Notice how Send doesn't take parameters, as it just sends the whole message history. We try to keep task responsibilities decoupled so you don't need to handle many similar tasks to handle some common/intercepted behavior.

If these tasks were emitted individually, then it would be difficult to inspect this kind of detail, and you might abort a single task that a future task was expecting to exist.

How do I leak data out of the hook closure?

Due to a hook being a closure executed by Chat at an unknown time for your parent widget, and because of Rust's lifetimes, we can't just directly access data from the outer scope of the closure.

So we need to communicate back with our parent widget somehow. We will not cover this with detailed examples as you are probably familiar with how to communicate Makepad widgets, but here are some ideas of what you can do.

Since you have access to cx inside the closure, one way would be to emit an action with cx.widget_action(...) and receive it in your parent widget or use cx.action(...) and handle it globally.

If you don't want to define messages and instead you want to directly execute something in the context of your widget, you can use Makepad's UiRunner to send instructions packed in a closure back to your parent widget.

What interactions can I intercept with this?

Any interaction listed and documented in the ChatTask enum can be worked with from inside the hook.

You may want to read that enum's specific documentation in the crate documentation.

Web support

Prerequisites

This guide assumes you have already read the Quickstart.

How to

Moly Kit has been designed with web support from day one. If you are targeting only the web, you simply need to do the following:

Add wasm_bindgen to your app.

cargo add wasm_bindgen

Then, ensure you use its prelude somewhere. Let's just place it in the main file.

use wasm_bindgen::prelude::*;

fn main() {
    your_application_lib::app::app_main()
}

Finally, to run your app, you will need to do it like this:

cargo makepad wasm --bindgen run -p your_application_package

Note: You will need to have the cargo makepad CLI installed. Check Makepad's documentation for more information.

A warning about wasm-bindgen support in Makepad

By default, Makepad uses its own glue code to work in a web browser and doesn't work with wasm-bindgen out of the box.

The --bindgen argument we passed to cargo makepad earlier is very important as it enables wasm-bindgen interoperability in Makepad.

Targeting both web and native (non-web)

If you want to be able to build your app for both web and non-web platforms, you can adapt your main file to the following conditionally compiled code:

#[cfg(target_arch = "wasm32")]
use wasm_bindgen::prelude::*;

#[cfg(target_arch = "wasm32")]
fn main() {
    your_application_lib::app::app_main()
}

#[cfg(not(target_arch = "wasm32"))]
#[tokio::main]
async fn main() {
    your_application_lib::app::app_main()
}

Please notice how the main function targeting native platforms needs to run in the context of a Tokio runtime.

However, on the web, we can't use Tokio, as the browser has its own way of driving futures.

Advanced

This section covers advanced topics that are typically not necessary unless you need to extend Moly Kit beyond its built-in features. These chapters will guide you through more complex customizations and integrations.

Implement your own client

Prerequisites

This guide assumes you have already read the Quickstart.

Introduction

As we mentioned before, a "client" is something that allows us to interact with models/agents from providers like OpenAI, Ollama, etc., asynchronously.

In general, as long as Moly Kit has a client compatible with the provider you want to connect to, you don't need to implement your own client. For example, you may recall from the Quickstart that there is a built-in OpenAIClient that you can use with any OpenAI-compatible API.

You also have the MultiClient we mentioned in Use multiple providers, which is a utility client to compose multiple clients into one.

All these clients are clients because they implement the BotClient trait. This trait defines a set of (mostly) asynchronous methods to fetch and send data to providers.

If we want to build our own client, either to support an unsupported provider, to make a utility, or for any other reason, you simply need to implement that trait.

Implementing the BotClient trait

Some of the methods in this trait have default implementations, so we don't need to implement them, and they may not be relevant for us.

The ones that are very important to implement are the following:

struct MyCustomClient {
  // ... some config fields ...
}


impl BotClient for MyCustomClient {
    fn bots(&self) -> MolyFuture<'static, ClientResult<Vec<Bot>>> {
        let future = async move {
          // ... fetch our list of available models/agents ...
        };

        moly_future(future)
    }

    fn send(
        &mut self,
        bot_id: &BotId,
        messages: &[Message],
    ) -> MolyStream<'static, ClientResult<MessageContent>> {
        let stream = stream! {
          // ... write some code that yields chunks of the message content ...
        };

        moly_stream(stream)
    }

    fn clone_box(&self) -> Box<dyn BotClient> {
        Box::new(MyCustomClient)
    }
}

We can see from these implementation template a lot of details. First of all, the async methods return MolyFuture and MolyStream respectively. These are basically boxed dynamic futures/streams with some cross-platform considerations.

Note: Boxed dynamic futures/streams are one of the things that can make async code in Rust difficult to write. This is because this kind of box erases important type information that Rust would normally use to make our lives 10 times easier.

However, since generics will not play well with Makepad widgets when they reach the UI, dynamic dispatching is a necessary evil.

The next thing to notice is that we can very easily create those kinds of futures/streams from compile-time known futures by simply wrapping them with the moly_future(...) and moly_stream(...) functions.

We can also see these methods normally return a ClientResult, which is slightly different from Rust's standard Result as it can contain both successfully recovered data and multiple errors. However, the constructor methods of ClientResult enforce semantics similar to Result. If you use new_ok(...), it will contain just a success value. If you use new_err(...), it will contain just an error. And if you use new_ok_and_err, you are expected to pass a non-empty list of errors alongside your success data; otherwise, a default error will be inserted into the list, and you probably don't want that.

Let's talk a little more about what the methods do. The bots method simply returns a list of available models/agents. It's pretty simple to implement; you could use something like reqwest to fetch some JSON from your provider with the list of models, parse that, and return it.

send is the method that sends a message to a model/agent. It returns a stream which allows you to push chunks of content in real-time. Although, you could also yield a single chunk from it if you don't support streaming.

Different from Futures in Rust, Streams are a little less mature, so creating them with the async_stream crate is advised for simplicity.

Note that the exact implementation of a client greatly depends on the provider you are trying to support, so it's difficult to make a generic guide on how to build it step by step. I recommend checking how OpenAIClient and MultiClient are implemented as examples to create your own.

But to avoid leaving this section without a working client, let's finish the implementation with some dummy methods for a client that simply repeats the last message.

struct EchoClient;

impl BotClient for EchoClient {
    fn bots(&self) -> MolyFuture<'static, ClientResult<Vec<Bot>>> {
        let future = futures::future::ready(ClientResult::new_ok(vec![Bot {
            id: BotId::new("echo-provider", "echo-echo"),
            name: "Echo Echo".to_string(),
            avatar: Picture::Grapheme("E".into()),
        }]));

        moly_future(future)
    }

    fn send(
        &mut self,
        _bot_id: &BotId,
        messages: &[Message],
    ) -> MolyStream<'static, ClientResult<MessageContent>> {
        let last = messages.last().map(|m| m.content.text.clone()).unwrap_or_default();

        let stream = futures::stream::once(async move {
            ClientResult::new_ok(MessageContent {
                text: format!("Echo: {}", last),
                ..Default::default()
            })
        });

        moly_stream(stream)
    }

    fn clone_box(&self) -> Box<dyn BotClient> {
        Box::new(EchoClient)
    }
}

Try not to use tokio-specific code

Most relevant Tokio utilities are present in the futures crate, which is the base for most async crates out there (including Tokio).

Using Tokio inside your client would make it unusable on the web.

Of course, if your custom client deals with native-specific stuff like the filesystem, stdio, etc., then it may be reasonable.

Handle provider-specific content format

Prerequisites

This guide assumes you have already read Implement your own client.

Introduction

By default, Moly Kit displays MessageContent using the StandardMessageContent widget.

This widget can render common content types returned by current LLMs, such as simple markdown, thinking blocks, and citations from web searches.

However, a BotClient might interact with models or agents that return more complex or unique content than what Moly Kit currently supports.

Therefore, BotClients need a way to extend Moly Kit to render such unique content.

This is quite straightforward. Clients can implement the content_widget method, which allows them to return a custom UI widget to be rendered in place of the default content widget, whenever the method deems it appropriate.

However, due to Makepad's architecture, users of your client must also perform some "registration" steps for this to work.

In summary, the high-level steps are:

• Create a standard Makepad widget tailored to your content needs.
• Implement content_widget in your client. This method will create the widget using a template obtained by its ID.
• Instruct users of your client to register the widget manually, like any Makepad widget, using live_design(cx).
• Instruct users of your client to create a template in Makepad's DSL and insert the LivePtr to that template under the expected ID.

Detailed instructions with an example

Let's start by creating our custom content widget. This can be anything you need. For this example, we'll implement one that simply displays text in a Label:

use makepad_widgets::*;

live_design! {
    use link::theme::*;
    use link::widgets::*;

    pub MyCustomContent = {{MyCustomContent}} {
        // It's important that height is set to Fit to avoid layout issues with Makepad.
        height: Fit,
        label = <Label> {}
    }
}

#[derive(Live, Widget, LiveHook)]
pub struct MyCustomContent {
    #[deref]
    deref: View,
}

impl Widget for MyCustomContent {
    fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
        self.deref.draw_walk(cx, scope, walk)
    }

    fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
        self.deref.handle_event(cx, event, scope)
    }
}

impl MyCustomContent {
    pub fn set_content(&mut self, cx: &mut Cx, content: &MessageContent) {
        self.label(id!(label)).set_text(cx, &content.text);
    }
}

Note: Making a set_content method that takes MessageContent is just a convention. It's not strictly necessary. You can design it however you want.

The next step is to implement the content_widget method in your BotClient:

impl BotClient for MyCustomClient {
    // ... other methods implemented ...

    fn content_widget(
        &mut self,
        cx: &mut Cx,
        previous_widget: WidgetRef,
        templates: &HashMap<LiveId, LivePtr>,
        content: &MessageContent,
    ) -> Option<WidgetRef> {
        // We expect the user of our client to register a template with the
        // id `MyCustomContent`.
        let Some(template) = templates.get(&live_id!(MyCustomContent)).copied() else {
            return None;
        };

        let Some(data) = content.data.as_deref() else {
            return None;
        };

        // Let's assume `MessageContent` yielded from our `send` contains
        // this arbitrary data, explicitly stating it wants to be rendered with
        // `MyCustomContent`.
        if data != "I want to be displayed with MyCustomContent widget" {
          return None;
        }

        // If a widget already exists, let's try to reuse it to avoid losing
        // state.
        let widget = if previous_widget.as_my_custom_content().borrow().is_some() {
            previous_widget
        } else {
            // If the widget was not created yet, let's create it from the template
            // we obtained.
            WidgetRef::new_from_ptr(cx, Some(template))
        };

        // Let's call the `set_content` method we defined earlier to update the
        // content.
        widget
            .as_my_custom_content()
            .borrow_mut()
            .unwrap()
            .set_content(cx, content);

        Some(widget)
    }
}

Now, anyone who wants to use this client will need to register the widget like any normal Makepad widget:

impl LiveRegister for App {
    fn live_register(cx: &mut Cx) {
        makepad_widgets::live_design(cx);
        moly_kit::live_design(cx);

        // Add this line.
        my_custom_client::my_custom_content::live_design(cx);

        crate::widgets::live_design(cx);
    }
}

And finally, let's create a template for it and insert it into the Chat widget's Messages component.

use makepad_widgets::*;

live_design! {
    use link::theme::*;
    use link::widgets::*;

    use moly_kit::widgets::chat::*;
    use my_custom_client::my_custom_content::*;

    pub MyCoolUi = {{MyCoolUi}} {
        // Notice the `:` here instead of `=`, to bind to the `#[live]` property
        // below.
        my_custom_content: <MyCustomContent> {}
        chat = <Chat> {}
    }
}

#[derive(Live, Widget)]
pub struct MyCoolUi {
    #[deref]
    deref: View,

    #[live]
    my_custom_content: LivePtr,
}

impl Widget for MyCoolUi {
    fn draw_walk(&mut self, cx: &mut Cx2d, scope: &mut Scope, walk: Walk) -> DrawStep {
        self.deref.draw_walk(cx, scope, walk)
    }

    fn handle_event(&mut self, cx: &mut Cx, event: &Event, scope: &mut Scope) {
        self.deref.handle_event(cx, event, scope)
    }
}

impl LiveHook for MyCoolUi {
    // Let's insert the template as soon as the widget is created.
    fn after_new_from_doc(&mut self, _cx: &mut Cx) {
        // ... other initialization code ...

        let chat = self.chat(id!(chat));
        let messages = chat.read().messages_ref();

        // We must use the ID that `content_widget` expects.
        messages.write().templates
            .insert(live_id!(MyCustomContent), self.my_custom_content);
    }
}

And that's it! All four pieces are in place. Now, whenever content_widget returns Some(a_custom_widget), Messages (the widget that renders the list of messages inside Chat) will replace its default content widget with the custom one.