Code Deep Dive

Before Astro's build even starts

• Fetching all Notion Pages the NOTION_TOKEN (environment variable) has access to

• Transform 'raw' Notion pages to make them more usable on next steps.

• Populate pages with their blocks recursively

  • Exceptions for the following blocks: link_to_page, child_page, child_database. These will be filled a few steps later when pages get their full data.

  • We use the library notion-to-markdown to convert blocks to md then generate the markdown for the page. (This was a quick win but will be technical debt in the short term)

  • Each page is parsed with MDX as well so we can extract frontmatter and other exportsearly and use these as build options if necessary.

• Find the root of the project (settings page) with these rules. Either

  • the only Notion page (the token can access) placed at the root in Notion, or

  • the page corresponding to the ID provided in the NOTION_ROOT_ID environment variable

• Create a full tree of the Notion data by placing pages into their respective blocks of types link_to_page, child_page, child_database

• Transform the tree of Notion Pages to a more friendly data structure (and more Unist friendly hopefully)

  • 3 types of nodes:

    • root is the settings page = the root page in Notion. This node has a property data.role = "settings"

    • page: can be a classic page or a database page (the latter is not properly supported yet though) with respectively data.role = "page" and data.role = "collection"

    • blockfor Notion blocks. These are more or less equivalent to HTML block elements.

    • This is the current mapping of Notion block types into our node's data.role

      const rawTypes = {
      	paragraph: "p",
      	heading_1: "h1",
      	heading_2: "h2",
      	heading_3: "h3",
      	bulleted_list_item: "ul",
      	numbered_list_item: "ol",
      	code: "code",
      	to_do: "todo",
      	toggle: "toggle",
      	child_page: "page",
      	child_database: "collection",
      	embed: "embed",
      	image: "img",
      	video: "video",
      	file: "file",
      	pdf: "pdf",
      	bookmark: "bookmark",
      	callout: "callout",
      	quote: "blockquote",
      	equation: "equation",
      	divider: "hr",
      	table_of_contents: "toc",
      	column_list: "columns",
      	column: "column",
      	link_preview: "a",
      	synced_block: "skip",
      	template: "none",
      	link_to_page: "link",
      	table: "table",
      	table_row: "tr",
      	unsupported: "none",
      };
      
      

  • NOTE: Inline elements are currently described inside their block (as an 'Notion-style' array of data and as markdown). In the near future, we might want to transform this data in a proper subtree to allow easier composability and improve compatibility with the Unified ecosystem.

• Extract relevant info from the settings page.

  • globalStylesString will be injected in a user-styles.css thanks to the 'non-html page' Astro API. (See in /src/pages/user-styles.css.js)

  • headString will be injected into the head of each page with a <Fragment slot="head" set:html={headString} /> inside our <SkeletonPage ... /> layout.

• Complement page info:

  • Construct the full URL of the page based on own page name and parents' names (keeping '/' in place). For example, a structure in Notion like "root" > "Super" > "SeCrEt" > " / hidden / / SECRET - Page / " would create this path super/secret/hidden/secret-page

  • Merge MDX exports from root to each page. The data cascade seems the most convenient way to handle data to avoid repetition.

• Record a list of all pages with Notion ID, Notion URLs for this page and our generated path. This is usefull to be able to replace internal Notion links with our internal website's links.

• Process Images and files

  • We have 5 properties that can describe an image or file

    • featuredImage (only on nodes with type = 'page') is a page's icon (if the icon is an image. See further in case it is an Emoji)

    • cover (only on nodes with type = 'page') is a page's cover image.

    • icon (only on nodes with data.role = 'callout') is a callout's icon (if the icon is an image. See further in case it is an Emoji)

    • image (only on nodes with data.role = 'img') is the content of an independant img block

    • file (only on nodes with data.role = 'file') is the content of an independant file block

  • An image of file description has the following properties:

    const imageOrFile = {
    	last_edited_time, // comes directly from Notion
    	originalUrl, // Notion URL (expires for uploaded ressources)
    	filename, // like 'my-image.png'
    	extension, // like '.png'
    	url, // our local file URL
    }
    

• An emoji property can contain the emoji character chosen in Page Icon or Callout Icon.

• Process Notion's Page Properties (the properties attributed to a page when it is a child of a database)

  • ... This is a work in progress ...

  • We try to make this props as easily usable as possible on the front-end to make it easy for components to consume the data (as props since these are passed as props to the page).

  • For example, a property called 'country' defined as a single select can be accessed as props.country on the front-end and will return a single string (for example "Belgium")

  • For the moment, these are handled:

    • rich_text will be returned as plain string

    • select will be returned as single string

    • multi_select will be returned as array of strings

    • files will be returned as array of fileObject (as described above)

• Merge Notion page properties and MDX exports into data.props

• Record a list of all assets (images and files) with the same properties as described earlier. This is usefull to be able to replace Notion's generated links with our internal website's links.

• Download all assets locally (currently hard coded to public/user-assets) and unzip .zip files in place.

• Replace Notion URLs (pages and assets) in blocks data and in markdown.

• Isolate Settings for convenience

• Flatten Pages as an array for convenience

• Save computed data to disk for access during build and on the front-end.

  • A JSON file is created at src/_data/poko.json (currently hard coded)

  • The structure is the following:

    const poko = {
    	settings, // the isolated root node
    	pages, // array of pages with all info we gathered and full list of children recursively
    	files, // array of `fileObject` definitions for assets downloaded from Notion
    	paths, // array of `pathsMap` for all our pages
    	websiteTree, // the full tree of nodes from root (settings) to leaves (blocks on the most nested pages)
    }
    

Notes about 'fetching ahead'

This part is currently handled by a minimal Astro integration I called astro-fetch-ahead. The only role of this integration is to execute an async function before the actual build begins.

The goals of using this could be:

• to execute the code only once even on dev

• to run code early and potentially fail early too

• to play with meaningful Astro paths without interfering (like dumping assets in the public/ directory.

TODO: There is a potential for improving developer experience tenfold by running this in parallel to the dev server to live reload on changes in Notion.

The front-end mainly consists of

• A 'catch-all' [...path].astro page

  • Imports poko data

  • Re-process pages and settings with MDX + data cascade

  • Generate pages according to their paths

  • Dispatch the page to the desired Renderer Layout

• A SkeletonPage.astro layout

  • Handles metadata and head injection

• Default MDX components

  • ... Work in progress ...

  • A set of smart default components

  • Can be overwritten in code AND in Notion directly through exports

• Renderer layouts. Currently MDXPage.astro is the only useful one in all practicality.

  • Renders the desired flavour of Markdown and/or the data tree directly