Solving pain with references to draft documents in Sanity CMS. How to set up a full-fledged preview environment for your content

July 2023 Update. Sanity released a new feature - perspectives. It gives us a simple way of controlling what subset of documents will be fetched. With this feature enabled, we don't need to construct complicated queries anymore. It implements exactly the desired behavior but on the Sanity backend level. Read our fresh article with a comprehensive guide to Sanity's Perspectives.

At the end of 2021, the Sanity team made a significant breakthrough in solving the long-awaited issue with document referencing by releasing the "References in Place" feature. What opportunities for content modeling does this give us in 2023 and has the problem with references completely been solved?

The short answer is no, not completely.

However, now we have the opportunity to get the desired result by starting from new possibilities.

Problem statement

1. Content modeling for most of the projects is unthinkable without links between documents. In Sanity, there is a field type for this - reference
2. Also, modern headless CMSs assume publishing flow with at least two document states: draft and published. In Sanity, this feature is provided out of the box - you can edit draft documents, publish them and then re-change them again
3. We want to be able to preview content before publishing it. There is also such an opportunity - by filtering documents by their IDs detecting theirs belonging to drafts or published document versions.

So everything looks fine, can we start implementing it? No, not yet, actually. The problem lies in the fact that all together it does not work. The point is that we can't have references to draft versions of documents in Sanity Studio. If such a connection is necessary and you simply cannot get it, then your app is broken on the preview environment.

Previously it worked this way: you had to publish the document and only after that add a reference to it. But the published document went into production. Here is such a vicious circle.

So what has changed after the release of “References in Place” and how will it help solve our problem?

References in Sanity Studio

Previously draft documents were filtered out in the documents dropdown when you tried to create a reference. With the new feature “References in Place”, this behavior has changed. Now you can select an unpublished document from the list and create a reference to it. However, the _ref field will not contain the ID of the draft document, but its published version even if the document has never been published yet.

A reference always contains a published ID

You can read how and why it was implemented in the original post, the link to which is given at the beginning. It is worth doing this in order to get an idea of the challenges that Sanity's team had to overcome and how it was ultimately implemented. Also, their post describes what UX improvements for CMS users this gives.

However, we will focus more on data fetching.

So, if we have two draft documents of different types: Document A and Document B, and according to our schema Document A has a field called "reference" which type is a strong link to Document B, then we get the following statements.

Table 1. Document IDs and reference

Note that the entry with _id “fe835523-7186-4520” does not currently exist. Are you wondering how a strong link can point to a non-existent document? This is the essence of the new Sanity feature. If we open the document in the inspector, we will see that Sanity Studio itself has added the following fields to the reference object:

• weak: true. Don't worry, this is temporary only and will be removed as soon as Document A is published
• _strengthenOnPublish - This property tells Sanity Studio that the link will need to be strengthened when the document is published.

That is after we publish Document B and then Document A, our reference will take the regular, familiar to us form. Once published documents still cannot reference drafts, however, drafts themselves can be linked. We just need to learn how to extract these connections.

Table 2. Referencing ability

Spotted: An funny point is that if you decide to “unpublish” document B, then Sanity Studio will not return the link to its original state, but simply inform you about the impossibility of this operation.

Preview environment setup

Let's consider the simplest example of a schema containing documents of the type Post and the type Author. Post has a reference to an Author. You can get such a content model as a sample if you select "Blog (schema)" when you initialize a new project with the Sanity CLI.

Image 1. Select starter project

Obviously, we will show only published content on production. Let us have a public dataset, so that means the published content is available without a token, and only authed requests can query draft documents. To fetch information about posts and authors, we can write, for example, the following GROQ query:

Script 1. Simple GROQ query

* [_type == "post"] {

...,

author->

}
  

On production, this request will return exactly what we plan to show to users.

Let's also agree that we are not going to create separate versions of the application for production and preview environments. It should be the same code, behaving differently based on environment variables. In particular, in our case, this means that the GROQ query should be the same, and the difference will be in having or missing auth token, which gives us all or a published-only subset of document variants.

So how do we write a query that would give us the following result?

Table 3. Content environments

Suppose we have a new author and they wrote their first post. The editor-in-chief needs to preview and approve both documents (the author page and the post itself) prior to publication. For that, the editor-in-chief wants to open these pages on the preview. Each page contains information given from both documents.

Obviously, by executing the query from [Script 1], we will get all the posts from the dataset. However, the link author-> to the author will be empty, because at the moment it refers to a non-existent document.

Getting references

How can I get a draft of a document when I have a reference to the published version? Let's add the "drafts." suffix to the id.

Script 2. Querying draft reference

* [_type == "post"] {

...,

"author": * [_id == ("drafts." + ^.author._ref)][0],

}

  

Great, now the author field will contain the data of the draft author in the preview environment. However, this way we broke our app on production because, for published authors, the author field will be wrong. It simply will be empty as still trying to get a draft document.

Let's fix it properly.

Script 3. Getting correct variant

* [_type == "post"] {

...,

"author": coalesce( * [_id == ("drafts." + ^.author._ref)][0], author->)

}

  

Let's see what's going on here. The coalesce function returns the first non-null result. In production, the first argument will always be null, since drafts are automatically filtered out, and on the preview, if a draft exists, then according to Table 3 we should display it. Now our references link to the correct versions of the documents.

Getting documents

If some posts were published and then re-edited, then our GROQ query will return a list of posts including all available draft and published versions. Obviously, we don't want to show both versions of one post at once - the site should look the way it will be after all the documents are published. Let's find a way to filter documents the way we want.

The difficulty here is that by making a request for a document of a certain type, we get a flat list of all available variants of documents on that dataset. Literally, such a query will return an ID array containing both the id of the published documents and the drafts.

Script 4. Getting correct variant

* [_type == "post"]._id

Moreover, some id will appear in two variants at once: “drafts.xxx-yyy” and “xxx-yyy”. Complicating matters is the fact that Sanity does not directly link drafts and public documents to each other. This leads to the fact that having received any document, we cannot simply find out if it has another version. The intersection of published and draft document spaces is shown in Image 2.

Image 2. Subsets of published and draft IDs

The only way to find out if the received document has a complimentary variant is to generate that variant ID and check the document’s existence.

For example, this is how you can get a draft for each published document: "draft": * [_id == ("drafts." + ^._id)].

Let's write a query to get the required selection of documents.

Script 5. Selecting documents

{

"docs": * [_type == $type]

}

{

"drafts": @.docs [_id in path("drafts.**")],

"published": @.docs [!(_id in path("drafts.**"))],

"coupled": @.docs [!(_id in path("drafts.**"))]{"published": {...}, "draft": ^.docs [_id == "drafts." + ^._id][0]},

}

{

"allCoupled": [...@.coupled, ...@.drafts [!(@._id in ^.published[]{"_id": ("drafts." + _id)}._id)]{"draft": {...}}],

}

{

"selected": @.allCoupled[]{

...coalesce(@.draft, @.published)

},

}.selected[] {

...,

"author": coalesce( * [_id == ("drafts." + ^.author._ref)][0], author->)

}

  
  

Let's see what's going on here.

• docs is an array of all variants of documents of the type we need. If necessary, you can add additional conditions to the filter.
• drafts and published - here we divide all variants by published and draft IDs. Obviously, some documents have both versions, and some have only one.
• coupled - into this array we put objects describing full versions of documents that were ever published. Each such object contains two properties: "draft" and "published", where we're keeping the corresponding variants. If a document lacks a draft version, then the "draft" property will be null.
• allCoupled - here we add draft-only documents to the previous array. Thus, as a result, we have a complete list of documents with their variants.
• To create the selected array, we go through all the documents from allCoupled and select the variant we need. On production, these will only be published versions, and on preview environment - both published and drafts. If both variants are present, the draft will be selected.

For your convenience, all scripts given in this article are placed in a separate GitHub gist.

Disadvantages and Alternatives

The presented solutions allow you to achieve the desired result, but they are not ideal. Script 5 looks too cumbersome to use each time when you need to get a list of documents of a certain type. An alternative solution is to get all documents with a simple query and filter them using JavaScript.

Another problem is that you need to constantly keep this solution in mind when writing a new query. For example, when retrieving a document by reference using Script 3, we must not forget how we do it.

And here we come to a more global problem. Let's take the experience we have at FocusReactive. We work a lot with projects on the jamstck and related technologies. On especially large projects, we, being experts in Headless CMS, develop only the CMS part, while client applications are developed by other teams. While we do our best to document the CMS, we still have no control over the code written on the frontend side.

And all requests and post-processing just happen on their part. From this point of view, the following requirements for the CMS are becoming a priority for us:

1. Transferring the logic of content selection for a given env from the client to the CMS part.
2. Simplifying the complexity of GROQ queries written by front-end developers as much as possible

An ideal solution for us would be a case where any front-end developer, after a short onboarding and learning the documentation, based on simple and clear instructions, could solve their daily tasks. At the FocusReactive team, we have made quite a bit of progress in this direction by customizing Sanity Studio and using custom auth tokens.

This solution will hardly fit into a single article, but if you are interested, we disclose some implementation details in this publication Multi Environment publishing flow with Sanity CMS. In addition, you can always contact us if you need expertise in Sanity or another Headless CMS.

Conclusion

As developers, we would like to have a convenient and simple solution for this problem. In fact, the tasks of selecting both documents and links are reduced to the following:

• We split all available documents into pairs: draft and published. Such variants of one document should be explicitly connected. In some pairs, one component may be missing.
• We have a simple rule for choosing the right variant from a pair. This choosing logic can be dumb and simple and be based on the presence/availability of a draft version.

Unfortunately, the current implementation of GROQ does not provide us with a ready-made tool for solving this problem. But Sanity is an evolving product and developers are responsive to the needs of the community. Would you like to be able to solve similar problems through the built-in capabilities of GROQ? Perhaps the Sanity developers will hear us, and this will be the next "obvious feature" implemented by the Sanity team.