Static Generators, Powerful Pipelines and Wyam Websites

We rebuilt our website using a static site generator and, in the process, learned to appreciate the power of pipelines.

(blog post Markdown goes here)


When running the Wyam build, this Markdown file will be picked up with the other Markdown and/or `.cshtml` files, run through the Razor view engine, and the rendered HTML will be written into the output folder. Our website keeps a blog, a news feed, and the pages for services in separate folders. Each folder, including the top-level, gets a Pipeline in order to process the content files. 

## Pipelines
The more powerful features of Wyam is its use of a [pipeline](https://wyam.io/docs/concepts/pipelines) for generating the site. A pipeline is a series of modules executed in a sequence that result in output documents. If you're familiar with the [Cake build automation platform](https://cakebuild.net/) you'll be right at home here; if you're not, you'll be ready for Cake when you're done, as both use a very similar approach.

Here's an example of the pipeline used to generate the blog as part of this website:

Pipelines.Add( "Blog", ReadFiles("blog/**.md"), FrontMatter(Yaml()), If( @doc.FilePath(Keys.RelativeFilePath).Extension == ".md", Markdown() ), Razor(), WriteFiles(".html"), Meta("BlogFile", string.Format(@"blog/{0}", @doc.String("DestinationFileName"))), Meta("SitemapItem", (string)@doc["BlogFile"]), Branch( GenerateFeeds().WithRssPath(ctx=> "blog/feed.rss") .WithRdfPath(ctx=> null).WithAtomPath(ctx => null) .WithFeedCopyright(ctx=> "2007-2017 Endpoint Systems. All rights reserved.") .WithFeedTitle(ctx=> "Endpoint Systems Blog") .WithFeedDescription(ctx=> "APIs, XML, BizTalk and cloud development") .WithFeedLink(ctx=> "https://endpointsystems.com/blog/index.html") .WithItemPublished((doc, ctx) => doc["Published"]) WriteFiles()
)
);

Taking it from the top, this pipeline does the following:

- Creates a pipeline named "Blog"
- Picks up all Markdown files in the directory
- Checks for any YML files containing additional metadata
- Runs an [If](https://wyam.io/api/wyam.core.modules.control/if/) module to check for a condition to see if I managed to pick up any Markdown files. If I have, they are run through the [Markdown](https://wyam.io/api/wyam.markdown/markdown/) module. For all intents and purposes this is redundant considering all blog posts will be written into Markdown files, but it's a great demonstration of the versatility of the pipeline model being used. 
- Runs the Markdown files (now HTML) and any .cshtml files through the Razor templating engine
- Writes the files to an `.html` extension
- Pushes the HTML file names into the metadata using the [Meta](https://wyam.io/api/wyam.core.modules.metadata/meta/). Note we are adding to the path here instead of just using the [DestinationFileName](https://wyam.io/api/Wyam.Common.Meta/Keys/7BA36EA1) key
- Push a [SitemapItem](https://wyam.io/api/wyam.common.modules.contents/sitemapitem/) into the file's metadata using our previously-created BlogFile value. This will get picked up in a [Sitemap](https://wyam.io/api/wyam.core.modules.contents/sitemap/) module we call in our parent diretory, which as the name implies, generates a site map for the entire site
- Use a [Branch](https://wyam.io/api/wyam.core.modules.control/branch/) module to iterate through the files and generate the [RSS feed for the blog](https://endpointsystesm.com/blogfeed.rss) using the [RssFeed](https://wyam.io/api/wyam.feeds.syndication.rss/rssfeed/) module. Note that we also set several different feed properties, including making sure we use the `Published` metadata value for the 

## Metadata

As you may have surmised by now, there is a ton of metadata getting generated and passed around. If you're new to Wyam it can be difficult to comprehend what's already available vs. what you're looking for or may want to create on your own. The best place to start when trying to understand the metadata system is the [Keys](https://wyam.io/api/wyam.common.meta/keys/) class, which you can use to get or set core module properties with. There's also [FeedKeys](https://wyam.io/api/wyam.feeds/feedkeys/) for the [GenerateFeeds](https://wyam.io/api/wyam.feeds/generatefeeds/) module, [BlogKeys](https://wyam.io/api/wyam.blog/blogkeys/) for the blog [recipe](https://wyam.io/docs/concepts/recipes) (recipes are great if you're looking for something preconfigured for a special purpose, like a blog), [DocsKeys](https://wyam.io/api/wyam.docs/docskeys/) for the docs recipe, [HtmlKeys](https://wyam.io/api/wyam.html/htmlkeys/) for the various HTML processing modules, [GitKeys](https://wyam.io/api/wyam.git/gitkeys/) if you want to use the [GitCommits](https://wyam.io/api/Wyam.Git/GitCommits) and/or [GitContributors](https://wyam.io/api/Wyam.Git/GitContributors) modules within the pipeline to do something with your SCM, [SearchIndexKeys](https://wyam.io/api/wyam.searchindex/searchindexkeys/) for building a [SearchIndex](https://wyam.io/api/Wyam.SearchIndex/SearchIndex) for your site, and [CodeAnalysisKeys](https://wyam.io/api/wyam.codeanalysis/codeanalysiskeys/) for the code analysis modules which help generate documentation pages. 

To set your own metadata values, go to your `config.wyam` configuration file and, up before the pipeline code, you can add things like:

Settings[Keys.Host] = "endpointsystems.com"; Settings[Keys.LinkHideExtensions] = false; Settings[Keys.LinkHideIndexPages] = false; Settings[FeedKeys.Copyright] = "2007-2017 Endpoint Systems";


Or you can set your own, like this:

Settings["figaroSlogan"] = "Figaro: Real Document Storage for .NET Developers";

Settings["summary"] = "Endpoint Systems is a software and services firm helping customers with APIs, microservices, integration and cloud migration.";


When I want to access these in my Razor templates, I find these properties within the `@Model`:

@Html.Raw(@Model["figaroSlogan"])

```

Deploying

The best thing about deploying a static website is how cheap your options can go. While there are many free services available, we opted for Google Cloud Storage buckets to host our static sites. Not only is it easy to set up and dirt cheap for what you get, you can also take advantage of Multi-Regional Storage and have instant geographic redundancy across the entire United States where we deployed our storage.

Since our deployment needs are pretty simple at the moment, we use a .bat file containing our gcloud commands for uploading the files and setting the permissions for everyting deployed. As we finish revamping the Figaro and Figaro documentation websites, however, a formalized build process for Wyam static site CI/CD process will be built, and given the recent changes in VSTS (which we use to build all of the diferent Figaro packages we have to generate), we're going to be looking at using Cake going forward for all of our project needs. No doubt there will be some insights to share there as well.

Enjoy!