pages tagged latex

Build System for Novel Writing

Sun, 22 Jul 2012 21:43:35 -0400

Background

This makefile-based build system provides a bit of extra gloss/configuration for using Sphinx and sffms to build conventional manuscripts and websites for novel-like hypertexts.

Usage Notes

This makefile is derives from the default Sphinx makefile, with a number of tweaks that I've been making to this configuration:

The PDF build system is much abbreviated, and included in the main makefile. This is a bit less conservative than the default for this file, but it works just as well in most situations. The output is also much less verbose.
The build is a lot less noisy. Errors are still printed, which makes it great for automated build systems (i.e. continuous integration.)
Sphinx will build two sites, one, in an outline/ directory would store the outline and working notes of a story, while the other in a source/ directory would store the actual text of the novel. *In practice, I also seem to keep another notes/ directory around for less formal and more early stage note taking.
There's a post-processing script for sffms that makes section headers work right for my usual structure. Tweak or remove at your discretion.
You must install Sphinx and sffms and have them accessible to the build process.

Code

File: makefile

 SPHINXOPTS    = -c ./
 SPHINXBUILD   = sphinx-build
 PAPER        =
 BUILDDIR      = build
 SFFMS_CLEANUP = bin/sffms-cleanup
 TITLE = name.title-of-book.pdf

 # Internal variables.
 PAPEROPT_a4    = -D latex_paper_size=a4
 PAPEROPT_letter = -D latex_paper_size=letter
 ALLSPHINXOPTS  = -q -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS)

 .PHONY: html sffms dirhtml singlehtml ms outline outline-single outline-pdf help
 .DEFAULT_GOAL = help

 help:
    @echo "A build system for generating novels and outlines using 'Sphinx' and 'sffms.'"
    @echo "All build products are located within '$(BUILDDIR).' Some related scritps are in,"
    @echo "'bin/'. See <http://tychoish.com/code/novel-makefile/> for more information."
    @echo ""
    @echo "Targets:"
    @echo ""
    @echo "  outline - builds an html website of the outline."
    @echo "  outline-single - builds a single html page version of the outline."
    @echo "  outline-pdf - builds a pdf of the outline using Sphinx's LaTeX builder."
    @echo ""
    @echo "  html - builds a html site of the text, with each page in its own file."
    @echo "  dirhtml - builds a html site with each page an index.html, in a directory."
    @echo "  singlehtml - builds the text in a single page."
    @echo ""
    @echo "  ms - builds an sffms styled manuscript PDF named $(TITLE)"
    @echo "  ms-tex - builds and processes a sffms TeX file."
    @echo ""
    @echo "  clean - removes $(BUILDDIR)/"

 #
 # Outline Generation
 #

 outline:
    $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) outline $(BUILDDIR)/outline
    @echo "[OUTLINE] (html) build complete."
 outline-single:
    $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) outline $(BUILDDIR)/outline-single
    @echo "[OUTLINE] (single-html) build complete."
 outline-pdf:
    $(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) outline $(BUILDDIR)/outline-pdf
    @echo "[OUTLINE] pdf build complete."

 #
 # Sphinx (HTML) Build System - Novel
 #

 html:
    $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) source $(BUILDDIR)/html
    @echo "[HTML] build complete."
 dirhtml:
    $(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) source $(BUILDDIR)/dirhtml
    @echo "[DIR] build complete."
 singlehtml:
    $(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) source $(BUILDDIR)/singlehtml
    @echo "[SINGLE] build complete."

 #
 # Manuscript Build System - Novel
 #
 SFFMS_CLEANUP = bin/sffms-cleanup

 $(BUILDDIR)/sffms/index.tex:sffms
    @$(SFFMS_CLEANUP) $(BUILDDIR)/sffms/*tex
    @echo "[SFFMS] tex santized."
 $(BUILDDIR)/sffms/$(TITLE):$(BUILDDIR)/sffms/index.pdf
    @mv $< $@
    @echo "[SFFMS] pdf renamed. See: $@"
 sffms:
    $(SPHINXBUILD) -b sffms $(ALLSPHINXOPTS) source $(BUILDDIR)/sffms
    @echo "[SFFMS] tex generated."

 ms:$(BUILDDIR)/sffms/$(TITLE)
 ms-tex:$(BUILDDIR)/sffms/index.tex

 #
 # PDF Build System
 #

 PDFLATEXCOMMAND = TEXINPUTS=".:$(BUILDDIR)/sffms/:" pdflatex --interaction batchmode --output-directory $(BUILDDIR)/sffms/

 %.pdf:%.tex
    @$(PDFLATEXCOMMAND) $(LATEXOPTS) '$<' >|$@.log
    @echo "[PDF]: (1/3) pdflatex $<"
 #  @makeindex -s $(BUILDDIR)/latex/python.ist '$(basename $<).idx' >>$@.log 2>&1
 #  @echo "[PDF]: (2/3) Indexing: $(basename $<).idx"
    @$(PDFLATEXCOMMAND) $(LATEXOPTS) '$<' >>$@.log
    @echo "[PDF]: (2/3) pdflatex $<"
    @$(PDFLATEXCOMMAND) $(LATEXOPTS) '$<' >>$@.log
    @echo "[PDF]: (3/3) pdflatex $<"
    @echo "[PDF]: see '$@.log' for a full report of the pdf build process."

 #
 # Meta
 #

 clean:
    rm -rf build

The following file, used above, does a little bit of post-processing that makes using sffms with longer more structured works possible. Use, or not, as you wish:

File: bin/sffms-cleanup

 #!/bin/zsh

 if [ "`uname`" = "Linux" ]; then
    SED_ARGS='-i -r'
 elif [ "`uname`" = "Darwin" ]; then
    SED_ARGS='-i "" -E'
 else
    echo "You're probably using an unsupported system. Do some testing, edit this file, and try again."
    exit 1
 fi

 SFFMS_FILE=$1

 while [ `pcregrep -c -M "^.chapter(.*)\{(.*)\}\n+.newscene" $SFFMS_FILE` -gt 0 ]; do
     sed $SED_ARGS ':a;N;$!ba;s@(.)chapter.*\{(.*)\}\n{2}.newscene@\1chapter{\2}@m' $SFFMS_FILE
 done

A LaTeX Build System

Sat, 14 May 2011 13:18:20 -0400

I've been throwing around the idea of creating a LaTeX-based document production service for people and work groups that produce a lot of paper documents. I think lawyers, academics, and PR-types are ideal examples, but I think the basic approach could be extrapolated to other kinds of work. The core of this proposal is that it takes a traditionally unstructured domain (document editing and generation,) and adds structure and infrastructure to provide software and service

LaTeX is a great system for making great documents: they look great and it's easy to build templates and styles so that your documents always look consistent. These features are the result of LaTeX's structure. The downside is the LaTeX tool chain is pretty big and editing is in a source format which may be a bit complicated for the uninitiated.

Components

Each of the following sections explain one of the basic components of such a system, along with a basic idea for a solution, and some rationale of why I propose each aspect of the "system."

To be fair, I envision this as a lose collection of associated tools, rather than a single application. "Unix-like" or "modular" as you please.

Build System

As a web service, you need something that will take input files that include bibliographies or citation lists, auxiliary style specifications, and source files, and output PDFs (and possibly HTML.) One of the ways to make LaTeX usable, I think, is to take out all of the idiosyncrasies around the build process and have a stable, and consistent build environment. This gets rid of weird situations where you have to intuitively know when you have to run LaTeX four times. Between cutting out the lengthy installation and configuration process, and simplifying the build workflow, the win is enormous.

While there's no reason that one couldn't develop such a tool to run on a single machine, the dependencies would be complicated, and if you were deploying this for a company (say,) it makes sense to centralize the build process (and templates) and as long as the build service/infrastructure is freely available, there's no functional reason to not have it as a service.

In terms of implementation, I think there needs to be two main interfaces. The first is a website that has a box that you drag files onto, and then generates a PDF and lets you download that as a file. The second is a RESTful-style interface that can be easily integrated with text editors so that you can issue a command and have a PDF open a few moments later as if you ran pdflatex locally. I can imagine other nifty extensions of this basic "pipe" framework. LaTeX-by-email, document sharing, personal document libraries, and so forth.

Also, I think it ought to go without saying that the system should use pybtex and YAML formating. The source files themselves should basically be standard LaTeX, but I think each document should have a YAML header for configuration information for the build process. With some optional flags in the header, one could also configure the server provide some level of pre-processing to make the LaTeX a bit easier to manage.

Editing Environment

A great editing environment for LaTeX (and really anything,) can make all the difference in the world between a good editing experience. While I envision the basic files to be basic LaTeX files with a YAML header (possible providing a user-configurable level of pre-processing, to make some mundane aspects of LaTeX easier to handle,) I think that it'd be best if the system included a pre-configured editing system.

My inclination is to use emacs as a basic framework or foundation for the editing platform. Take the basic emacs that we all know and love, and wrap it all up with an installer, some basic AUCTeX, configuration, yasnippet, some glue for the build service, and a few commands and menu-bar items and bundle it all together. The end result is an editing environment that is potentially really powerful and pretty easy to customize for specific uses. The best part is that all of the hard work of making an editor is done. How cool is that?

Tempting Structure

The thing I learned from using LaTeX was how to create an use templates to achieve more consistent products with less work all around. The first document you produce with LaTeX takes forever and can be pretty painful. Every document thereafter is perfectly identical and delightfully simple to produce. By controlling the build system and the editing environment, it becomes crazy easy to do awesome things with templates. In addition to simplifying some syntactic forms with the build system, the build system can become a platform for offering access to templates. Providing and managing templates in this way is really the crown of the whole system, because:

templates allow users, primarily organizations, to produce highly consistent documents.
templates allow document creators to focus on content rather than format.
templates allow edits to existing documents to be made more quickly and cleanly without affecting document style.
templates allow content to be re-purposed into multiple forms easily. The same content can form a memo or letter and a presentation, which further reduces the amount of work required to produce collateral.

Value

In general the value can be summarized as: better looking documents, more coherent workflows, with less work per-document, after a reasonable learning curve. Information workers in some industries and professions will find more value from systems like these: media, law, public relations, and academia spring to mind as fertile grounds for this kind of technology. At the same time, every industry produces documents and collateral of some kind and having better publication tools is useful everywhere.

Resources

I've compiled a few links links to projects that are doing complementary and related work.

Covered In LaTeX

Tue, 31 Aug 2010 00:00:00 -0400

Although I haven't used LaTeX much in the past few years, it was one of the primary tools that hastened my shift to using GNU/Linux full time. Why? I'd grown sick of fighting with document preparation and publishing systems (e.g. Microsoft Word/Open Office), and had started using LaTeX on my Mac to produce all of my papers and documents that needed to be output to paper-formats. Why switch? Because after a certain point of having every tool you use be Free software (because it's better!), it becomes easier and more cost effective to just jump the gun and by commodity hardware and use a system that's designed to support this kind of software (managing a large selection lots of free software packages on OS X can become cumbersome).

So why LaTeX? What's the big deal? Why do I care now? Well...

LaTeX is a very usable front-end/set of macros for the TeX typesetting engine. Basically, you write text files in a particular way, and then run LaTeX (or pdflatex) and it generates the best looking PDF in the world of your document. You get full control over things that matter (layout, look and feel) and you don't have to worry about things that ought to be standard (titles, headlines, citations with BibTeX, page numbering, hyphenation). The best part, however, is that once you figure out how to generate a document correctly once, you never have to figure it out again. Once you realize that most of the things you need to output to paper are in the same format, you can use the same template and be able to generate consistently formated documents automatically. There's a "compile" step in the document production process, which means changes aren't often immediately recognizable, but I don't think this is a major obstacle.

Word processing and document preparation is a critical component of most common computer users. At least, I'd assume so, though I don't have good numbers on the subject. In any case, I think it might be an interesting project to see how teaching people how to use LaTeX might both improve the quality of their work, and also the way that they're able to work. It's advanced, and a bit confusing at first, but I'd suspect that once you got over the initial hump LaTeX presents a more simple and consistent interface: you only get what you tell it to give you and you only see the functionality that you know about. This might make the discovery of new features more difficult, but it doesn't limit functionality.

I'm not sure that this post is the right space to begin a lesson or series on getting started with LaTeX, but I think as a possible teaser (if there's interest) that the proper stack for getting started with LaTeX would consist of:

A TeXlive distribution. You need the basic tool kit including pdflatex, TeX, Metafont, LaTeX, and BibTeX.
A Text Editor with LaTeX support: emacs, TextMate, etc. Plain text can be difficult and cumbersome to edit unless you have the right tools for the job, which include a real text editor.
Some sort of macro or snippet expansion system. TeX is great. But it's also somewhat verbose, and having an easy way to insert text into your editing environment, both for templates but also for general operations (emphasis, notes, etc.) is incredibly useful, and reduces pain greatly.
A template management system. This probably needn't be a formal software system, but just something to organize and store the basic templates that you will use.

And the rest is just learning curve and practice. Onward and Upward!