The result will look similar to this (try it out by clicking on the chat bubble—it's trained on the Motif docs):
The component can be used as follows:
<Promptfiles={props.files}placeholder="Ask Acme docs..."iDontKnowMessage="Sorry, I don't know." />
Optionally, you can use your own button component using the Icon
prop:
<Promptfiles={props.files}placeholder="Ask Acme docs..."iDontKnowMessage="Sorry, I don't know."Icon={<button>Ask Acme docs</button>} />
Training the model
First, install the DocsGPT integration. To do so, head over to the project settings, select Integrations → DocsGPT and tap "Install integration".

Now, open the Share menu, navigate to the Integrations tab, find the DocsGPT integration and tap "Train".

Testing the model
Let's test that the model works as expected by running some sample queries in the GPT Playground. In the Share menu, under the Integrations tab, tap "Open playground":

You can now prompt the model, and check that the answers match. The prompt also includes pointers to the most relevant pages used to produce the answer.
Querying the model via API
Now that the model is trained, we can programmatically query it, which is what we need to build our custom prompt interface. The endpoint is accessible at:
POST /api/motif/v1/integrations/gptsearch/completions
Behind the scenes, the response is served via a Vercel Edge function as a ReadableStream object. This improves the perceived responsiveness of the prompt, as tokens are being returned as soon as they are computed, and so users don't have to wait until the full response has been computed.
In addition to the tokens making up the prompt response, as part of the stream we also return a list of page IDs corresponding to the content that was provided as context for the prompt. This comes as the first part of the stream, and is delimited by the ___MOTIF_START_STREAM___
token; after that, the answer is streamed. So the stream looks as follows:
[<page_id_1>,<page_id_2>,...]___MOTIF_START_STREAM___The beginning of the answer...
Aside: How does the prompt work?
Under the hood, the training works as follows:
- All your pages marked as "Public" are sent to training. Private pages are not included, so the prompt will only be able to answer questions based on publicly available content.
- Each page is split into sections. A section is a part of your page content that starts with a heading (
#
,##
, etc.). Any Markdown content is included, such as images or code blocks, but currently, custom components (such as a tab bar) are omitted. - For each section, we create an embedding. This is a numerical vector that carries the "meaning" of your content. We use OpenAI's
text-embedding-ada-002
model for creating the embeddings.
Once the training is done, that is, once all the embeddings have been created, we are ready to answer questions. This is done as follows:
- When the user submits a question, we build an embedding similar to what we did with sections during training, and compare it to the stored embeddings. This gives us a ranking of the sections which are most "similar" to the question.
- We pick the top sections, and build a query of the form:Given the following sections, answer the question using only that information. If you are unsure and the answer is not explicitly written, say "Sorry, I am not sure how to answer that".Context sections:TOP_SECTIONSQuestion: USER_QUESTIONAnswer as Markdown (including related code snippets if available):
- Then we send this query to the OpenAI completions endpoint, using
text-davinci-003
. The answer is provided token-by-token as a ReadableStream. - Note: One thing to keep in mind is that, in the current version, if a section is too long (roughly, 500+ words), it will be omitted from the prompt. An easy way to address this is by cutting up a long section into smaller sections split up by subheadings (e.g.
###
).
Building the prompt interface
It's time to bring it all together! Let's create a file named prompt.tsx
in the components
folder, and copy the following code:
import cn from "classnames";import type { FC, ReactNode, SyntheticEvent } from "react";import {Fragment,useCallback,useEffect,useMemo,useRef,useState,} from "react";import { Dialog as HUIDialog, Transition } from "@headlessui/react@1.7.8";const API_URL = "/api/motif/v1/integrations/gptsearch/completions";const I_DONT_KNOW = "Sorry, I am not sure how to answer that.";const ChatIcon = ({ className }: { className?: string }) => (<svgfill="none"viewBox="0 0 24 24"strokeWidth={1.7}stroke="currentColor"className={className}><pathstrokeLinecap="round"strokeLinejoin="round"d="M8.625 9.75a.375.375 0 11-.75 0 .375.375 0 01.75 0zm0 0H8.25m4.125 0a.375.375 0 11-.75 0 .375.375 0 01.75 0zm0 0H12m4.125 0a.375.375 0 11-.75 0 .375.375 0 01.75 0zm0 0h-.375m-13.5 3.01c0 1.6 1.123 2.994 2.707 3.227 1.087.16 2.185.283 3.293.369V21l4.184-4.183a1.14 1.14 0 01.778-.332 48.294 48.294 0 005.83-.498c1.585-.233 2.708-1.626 2.708-3.228V6.741c0-1.602-1.123-2.995-2.707-3.228A48.394 48.394 0 0012 3c-2.392 0-4.744.175-7.043.513C3.373 3.746 2.25 5.14 2.25 6.741v6.018z"/></svg>);type DialogProps = {isOpen: Boolean;onClose: () => void;children: ReactNode;};const Dialog: FC<DialogProps> = ({ isOpen, onClose, children }) => {return (<Transition show={isOpen} as={Fragment}><HUIDialog open={isOpen} onClose={onClose}><Transition.Childas={Fragment}enter="ease-out duration-300"enterFrom="opacity-0"enterTo="opacity-100"leave="ease-in duration-200"leaveFrom="opacity-100"leaveTo="opacity-0"><div className="z-50 fixed inset-0 bg-black/20 backdrop-blur-md" /></Transition.Child><Transition.Childas={Fragment}enter="ease-out duration-200"enterFrom="opacity-0 scale-95"enterTo="opacity-100 scale-100"leave="ease-in duration-200"leaveFrom="opacity-100 scale-100"leaveTo="opacity-0 scale-95"><div className="z-50 fixed inset-y-0 inset-0 flex items-center justify-center"><HUIDialog.Panel className="relative w-full bg-white dark:bg-neutral-900 rounded-lg max-w-screen-sm">{children}</HUIDialog.Panel></div></Transition.Child></HUIDialog></Transition>);};const LoadingDots = ({ className }: { className?: string }) => {return (<span className="loading-dots"><span className={className} /><span className={className} /><span className={className} /></span>);};type AnswerProps = {answer: string;onLinkClick: () => void;};const Answer: FC<AnswerProps> = ({ answer, onLinkClick }) => {const [plugins, setPlugins] = useState<any>([]);const [ReactMarkdownComp, setReactMarkdownComp] = useState<any>(undefined);useEffect(() => {import("https://esm.sh/remark-gfm@3.0.1").then((mod) => mod.default).then((gfm) => {setPlugins([gfm]);});}, []);useEffect(() => {if (!plugins) {return;}import("https://esm.sh/react-markdown@8.0.5").then((mod) => mod.default).then((RM) => {setReactMarkdownComp(<RMremarkPlugins={plugins}components={{a: (props: any) => {return <a {...props} onClick={onLinkClick} />;},}}>{answer}</RM>);});}, [answer, plugins]);return (<div className="flex flex-col gap-4"><div className="prose dark:prose-invert">{ReactMarkdownComp}</div></div>);};type PathMeta = {path: string;meta?: { title: string } & { [key: string]: any };};type ReferenceInfo = {path: string;title: string;};type IdPathMetaMap = { [key: string]: PathMeta };type ReferencesProps = {references: string[];idPathMetaMap: IdPathMetaMap;onLinkClick: () => void;};const References: FC<ReferencesProps> = ({references,idPathMetaMap,onLinkClick,}) => {const referenceInfo = useMemo(() => {return (references || []).slice(0, 5).map((id) => {const pathMeta = idPathMetaMap?.[id];if (!pathMeta) {return undefined;}const title = pathMeta.meta?.title || "Untitled";const path = pathMeta.path ?? "/";return { path, title };}).filter(Boolean) as ReferenceInfo[];}, [references]);return (<>{referenceInfo.length > 0 && (<div><p className="font-medium mb-1">References</p>{referenceInfo.map(({ path, title }) => {return (<aclassName="block subtleUnderline text-sm text-neutral-500 dark:text-white/50"href={path}onClick={onLinkClick}>{title}</a>);})}</div>)}</>);};type DialogContentProps = {idPathMetaMap: IdPathMetaMap;onLinkClick: () => void;placeholder?: string;iDontKnowMessage?: string;};const DialogContent: FC<DialogContentProps> = ({idPathMetaMap,onLinkClick,placeholder,iDontKnowMessage: _iDontKnowMessage,}) => {const [prompt, setPrompt] = useState<string | undefined>(undefined);const [answer, setAnswer] = useState("");const [references, setReferences] = useState<string[]>([]);const [loading, setLoading] = useState(false);const answerContainerRef = useRef<HTMLDivElement>(null);const iDontKnowMessage = _iDontKnowMessage || I_DONT_KNOW;const submitPrompt = useCallback(async (e: SyntheticEvent<EventTarget>) => {e.preventDefault();if (!prompt) {return;}setAnswer("");setReferences([]);setLoading(true);try {const res = await fetch(API_URL, {method: "POST",headers: {"Content-Type": "application/json",},body: JSON.stringify({ prompt, iDontKnowMessage }),});if (!res.ok || !res.body) {// Don't show the verbatim error message to users, but print// it in the console.console.warn(await res.text());setAnswer(iDontKnowMessage);setLoading(false);return;}const reader = res.body.getReader();const decoder = new TextDecoder();let done = false;let startText = "";let didHandleHeader = false;let __references = [];while (!done) {const { value, done: doneReading } = await reader.read();done = doneReading;const chunkValue = decoder.decode(value);if (!didHandleHeader) {startText = startText + chunkValue;if (startText.includes("___MOTIF_START_STREAM___")) {const parts = startText.split("___MOTIF_START_STREAM___");try {__references = JSON.parse(parts[0]);} catch {}setAnswer((prev) => prev + parts[1]);didHandleHeader = true;}} else {setAnswer((prev) => prev + chunkValue);}}setReferences(__references);} catch (e) {console.warn("Error", e)setAnswer(iDontKnowMessage);}setLoading(false);},[prompt]);useEffect(() => {answerContainerRef.current?.scrollIntoView({ behavior: "smooth" });}, [answer]);return (<div className="absolute px-5 py-4 flex flex-col inset-0"><div className="flex-none w-full"><form onSubmit={submitPrompt}><div className="relative"><div className="absolute inset-y-0 left-0 flex items-center pl-2">{loading ? (<LoadingDots className="bg-neutral-500 dark:bg-white/50" />) : (<ChatIcon className="w-5 h-5 text-neutral-500 dark:text-white/30" />)}</div><inputvalue={prompt || ""}onChange={(e) => setPrompt(e.target.value)}placeholder={placeholder || "Ask a question..."}className="w-full block py-2 pl-11 pr-16 text-sm text-neutral-900 dark:text-white/80 placeholder:text-neutral-400 dark:placeholder:text-white/50 focus:outline-none sm:text-sm transition bg-transparent"type="text"/><div className="absolute inset-y-0 right-0 flex items-center pr-2"><div className="text-xs text-neutral-400 dark:text-white/20 border border-neutral-200 dark:border-white/10 rounded px-2 bg-neutral-50 dark:bg-white/10">Esc</div></div></div></form></div><div className="flex-grow mt-2 py-6 h-full overflow-y-auto border-t border-neutral-100 dark:border-white/5"><Answer answer={answer} onLinkClick={onLinkClick} /><divclassName={cn("mt-8 transition duration-500", {"opacity-0": !references || references?.length === 0,})}><Referencesreferences={references}idPathMetaMap={idPathMetaMap}onLinkClick={onLinkClick}/></div><div className="h-24 w-full" /><div ref={answerContainerRef} /></div><p className="pt-4 pb-1 border-t border-neutral-100 dark:border-white/10 text-xs text-neutral-400 dark:text-white/20">Powered by{" "}<aclassName="underline"href="https://motif.land"target="_blank"rel="noreferrer">Motif</a>{" "}and{" "}<aclassName="underline"href="https://openai.com"target="_blank"rel="noreferrer">OpenAI</a></p></div>);};type FileTree = {files: FileTree[];folders: FileTree[];id: string;path: string;name: string;meta: { [key: string]: any };};const toIdPathMetaMap = (tree: any) => {let saveMap: any = {};const updateMap = (tree: FileTree) => {for (const file of tree.files) {const name = file.name.split(".").slice(0, -1).join(".");saveMap[file.id] ={ path: file.path, name, meta: { title: name, ...file.meta } } || {};}for (const folder of tree.folders) {updateMap(folder);}};updateMap(tree);return saveMap;};type PromptProps = {files: FileTree;placeholder?: string;iDontKnowMessage?: string;Icon?: ReactNode;};export const Prompt: FC<PromptProps> = ({files,placeholder,iDontKnowMessage,Icon,}) => {const [isOpen, setIsOpen] = useState(false);const idPathMetaMap = useMemo(() => {return files ? toIdPathMetaMap(files) : {};}, [files]);useEffect(() => {const onKeyDown = (event: any) => {if (event.key === "k" && (event.metaKey || event.ctrlKey)) {event.preventDefault();setIsOpen(true);}};window.addEventListener("keydown", onKeyDown);return () => {window.removeEventListener("keydown", onKeyDown);};}, []);return (<><divonClick={() => setIsOpen(true)}className={cn("w-min group transition rounded-md cursor-pointer", {"p-2 hover:opacity-90 hover:bg-black/5 dark:hover:bg-white/5": !Icon,})}>{Icon ?? (<ChatIcon className="w-5 h-5 text-neutral-900 dark:text-white/50 transition" />)}</div><Dialog isOpen={isOpen} onClose={() => setIsOpen(false)}><div className="max-w-screen-sm mx-auto h-[calc(100vh-120px)] max-h-[720px] overflow-hidden"><DialogContentidPathMetaMap={idPathMetaMap}onLinkClick={() => setIsOpen(false)}placeholder={placeholder}iDontKnowMessage={iDontKnowMessage}/></div></Dialog></>);};
In addition, copy the following CSS to /styles/main.css
:
.loading-dots {@apply inline-flex text-center items-center leading-7;}.loading-dots > span {@apply rounded-full w-1.5 h-1.5 !important;animation-name: blink;animation-duration: 1.4s;animation-iteration-count: infinite;animation-fill-mode: both;margin: 0 2px;}.loading-dots > span:nth-of-type(2) {animation-delay: 0.2s;}.loading-dots > span:nth-of-type(3) {animation-delay: 0.4s;}@keyframes blink {0% {opacity: 0.2;}20% {opacity: 1;}100% {opacity: 0.2;}}.subtleUnderline {text-decoration: underline dotted #d4d4d8;}
That's it! We now have a prompt interface ready to serve custom answers from an OpenAI language model trained on all our public pages! It can be used as follows in an MDX page:
import { Prompt } from "@components/prompt"<Prompt files={props.files} placeholder="Ask Acme docs..." />
Note the props.files
passed in the files
prop. This enables the component to bind the reference file IDs to page paths and provide readable links in the response.
When used in a template, use the files
prop available inside the template:
import { Prompt } from "@components/prompt"export const Template = ({ files, children }) => {return (<div><Prompt files={files} placeholder="Ask Acme docs..." />{children}</div>)}