Authorea and Github

Objective

I am writing a joint white paper with two collaborators. One collaborator writes primarily with LaTeX and BibTeX while the other collaborator writes primarily with MS Word and Endnote. I am a LaTeX/BibTeX person; my BibTeX database of papers has over 3362 entries.

There are presently three good options for writing LaTeX collaboratively: ShareLaTeX [1], Overleaf [2], and Authorea [3]. Authorea has a number of features that look attractive for writing collaboratively. It has a web interface that can handle input in LaTeX, Markdown, and plain text format. It has really nice helper functions for adding references starting from a doi number. Finally, it looks to have good Github integration.

I already have a GitHub account. So I sign up for Authorea. Under the free plan I get one private document, which I will use to compose the white paper collaboratively.

First Impressions of Authorea

I play around by entering text and comments in Authorea. Here are my first impressions and observations:

• Bibliography. I started with one of their document templates. The template contains two references. I deleted the whole paragraph, including the references. Yet the associated citations still show up in the bibliography. This seems like a bug. Try to chat online about cleaning up the bibliography using the chat feature in Authorea. The responders are offline, so the chat window reports that it is sending the query in an email instead. Wait and see.

• Subdocuments. Enter text in the article's Document > Main View and look what changes in the Document > Data Folder view. If I insert a LaTeX block in the Main View, then the associated data is stored internally as a .tex file in the Data Folder. If I insert a text block in the article, in contrast, then the associated data is stored internally as an .html file. The layout.md file contains a list of the files containing the articles text. After entering text in both a LaTeX window and a text window, it now looks like

  untitled.tex
  uhrbs9bs2js.tex
  5onp96q5068.html
  

  which indicates that my document presently consists of two LaTeX-style paragraphs and one text-style paragraph. If I try to compile the document in LaTeX, how do I incorporate the text buried in the text-style (i.e., .html file) paragraph? I conclude that if I'm going be doing final assembly of the document offline with a LaTeX compiler, then I need to instruct my collaborators to write using only LaTeX blocks.

• Download as ZIP. If I download the paper as a zip archive, then the contents of the LaTeX-formatted (.tex) and text paragraphs (.html) get converted to LaTeX and dumped to a single LaTeX file, full_article.tex in the .zip archive. I have tested this conversion in the case where the text paragraph contains a citation. I have not yet tested it with a figure.

• Commenting. Comments seem to be anchored to the paragraph and not to highlighted text within the paragraph. So if I comment then I should probably explicitly mention the text within the paragraph that you are commenting on. Will need to tell collaborators to do this too.

• Editing online. Editing in a text window in a browser is buggy. In a LaTeX-style paragraph, I cannot enter text following a reference if the text window ends in a reference. Try deleting the offending reference at the end of the text window. It just won't let me. The only way I can get the reference to go away is to go into the associated .html file in the Data Folder and remove the associate html code by hand. Arrgh.

Syncing with GitHub

See Authorea's memo Syncing articles to GitHub [4]. Log into Authorea and GitHub. In Authorea (my private document, Main View), click the gear in the upper right hand corner of the screen, and pull down to Github. I am going to select [Setup Github integration automatically]. Now I am asked

Authorea by natejenkins
wants to access your JohnMarohn account

    Personal user data
    Full access

    Repository webhooks and services
    Admin access

    Repositories
    Public and private

Gulp. Click the green [Authorize natejenkins] button at the bottom. This dumps me back to the article. There is a message at the bottom which tells me that article is synced to GitHub.

Looking in GitHub, I see a new repository called

Microstructurally-Precise-Lead-Halide-Perovskites-Rational-Design-In-Situ-Characterization-and-Funct

Ouch, what a long name. In the repository, click the Settings tab and change the repository name to

_JAM_GT__Marohn201803__DOE_HOIP_White_Paper

which is what I would call it on my computer. After renaming the repository, add a README.md file in GitHub. In Authorea, refresh the Data Folder page and observe that the README.md file magically appears. Conclude that Authorea and Github are still synched after I renamed the repository on Github. Nice.

Let's see about editing locally. Open up a terminal, switch to my Dropbox directory, and type

git clone https://github.com/JohnMarohn/_JAM_GT__Marohn201803__DOE_HOIP_White_Paper.git

then (I am running OS 10.13)

cd _JAM_GT__Marohn201803__DOE_HOIP_White_Paper
open .

to switch to the cloned directory and open up the directory in the OS Finder. Edit README.md, commit changes, and push. Verify that the README.md has been revised on GitHub and at Authorea.

Now edit one the .tex files; add an equation, commit, and push to propagate the change up to GitHub. The new equation is automatically propagated up to Authorea, essentially instantaneously. Except ... there is an error in the equation and it won't render. Fix the LaTeX in Authorea. When I click outside the paragraph the new text get committed. Now on the command line if I pull, the equation is updated in the local copy of the git repository too. Neato; the syncing works in both directions.

Clean up the repository locally

git rm cldooshgpvk.html
git rm 1gqcr37f5pk.tex
git mv i8cq1fnjvpc.tex outline.tex
git mv uhrbs9bs2js.tex background.tex

Edit layout.md from

untitled.tex
uhrbs9bs2js.tex
i8cq1fnjvpc.tex
cldooshgpvk.html

to

intro.tex
background.tex
outline.tex

Push changes to GitHub. On Authorea, the filenames are updated but the article is still rendered properly. Yea. Conclude that I can rename files and, as long as as I update the important layout.md file accordingly, Authorea remains happy.

Compiling the Document Locally

The paper is broken into small tex files locally. How to compile?

Background reading

• Authorea: Syncing articles to GitHub [4]. Quote: "In order to build your article locally when working offline we suggest to use this Python Script by our fellow Erik Tollerud" [6].
• Authorea: Can I use custom LaTeX macros, newcommands, and packages? [5]. Add custom packages and commands to the file header.tex. There is a large list of supported packages. I'm guessing that I need to stick to the list of supported packages because Authorea ultimately renders my LaTeX code to html or xml or something web friendly.

Download local_build.py from Github [6] and put it in upper directory of my local repository, _JAM_GT__Marohn201803__DOE_HOIP_White_Paper. Add two more files from a recently prepared grant of mine

local_build.py      % Erik Tollerud's python program
DOE.cls             % my LaTeX grants template file
bst/grants-ARO.bst  % my LaTeX bibliographic formatting file

Creat a file preamble.tex and edit its contents to read

\documentclass[pgs,endfloat]{../DOE}
\sisetup{obeyall}
\usepackage{ragged2e}

\addhead{DOE BES white-paper preproposal}{Clancy, Estroff, Marohn}
\bibliographystyle{../bst/grants-ARO}
\renewcommand*{\bibfont}{\footnotesize}

This file is read by local_build.py but is ignored by Authorea. Create a file header.tex and edit its contents to read

\newcommand{\msun}{\,\mathrm{M}_\odot}
\usepackage{siunitx}
\usepackage{natbib}
% \usepackage[super,sort&compress,numbers]{natbib}

This file is read by Authorea but is ignored by local_build.py. The contents of the file layout.md reads

intro.tex
background.tex
outline.tex

Copy the .gitignore file from a recently prepared grant. This file tells git to ignore all the intermediate LaTeX-created files. Add to the last line the directive authorea_build/* which tells git to ignore the directory where local_build.py will put the compiled authorea_paper.pdf file.

Compile with

python local_build.py --flatten

The compiled grant is

authorea_build/authorea_paper.pdf

which I click on manually to open. Success.

Fail, Recover, Fail

OK, the basics seem to work as advertised. Now on to references and figures. References first. Next step: upload my primary BibTeX file so I can cite while I write.

How to handle multiple bib files? Put my primary BibTeX file, jam99.bib, in the bibliography/ folder. This is a big file, 6.8 MB.

I now want to use citations from my .bib file in the text I am writing. Frustratingly, the local_build.py script does not handle multiple .bib files. I can work around this problem by opening up authorea_paper.tex and modifying the end of the file to read

\bibliography{../bibliography/biblio,../bibliography/jam99}{}

I can then do one last "by hand" compile and the authorea_paper.pdf comes out right. Push to GitHub and Authorea. The background.tex file now contains the text

\section{Background and Significance}

\emph{Relation to ongoing and prior work in the field}:

Hybrid organic-inorganic perovskites (HOIPs) are an impressive new class  of low-cost photovoltaic materials that have already been used to build solar cells with power-conversion efficiencies exceeding twenty percent.

Introduced by Snaith and co-workers \cite{Stranks2013oct,Hodes2013oct}

% \cite{Stranks2013oct,Hodes2013oct,Snaith2013nov};
%intriguing charge-transport properties \cite{Brenner2016jan};
%knowledge gaps \cite{Berry2015sep};
%hysteresis and operational stability \cite{Stranks2015may,Yang2017jun}.

Authorea can't handle these citations. Check that the modified files are there in Authorea. The article fails to load, and won't render. Maybe the problem is that Stranks2013oct appears in both .bib files. Let me therefore modify the last line to read

Introduced by Snaith and co-workers \cite{Stranks_2013} \cite{Hodes2013oct}

where Stranks_2013 is in one .bib file and Hodes2013oct is in the other. No luck. The article still fails to load.

Let's try to recover. In the Authorea Data window, edit background.tex to read simply

\section{Background and Significance}

Testing.

The file renders OK in the Data window, but if I move to the Article Window, still no luck. I see a "loading article" message, then after a few minutes I get a "Failed to load article data" error message. Backup some more: from the Authorea Data window, delete background.tex entirely. Note that layout.md now consists of

intro.tex
outline.tex

So the file layout.md was automatically updated. Going to the Article window, still get a "Failed to load article data" error message. Back up even more: delete bibliography/jam99.bib. That solved the problem; now the article renders.

Try to add a large bib file again. Do this by manually uploading jam99.bib to the bibliography/ folder by clicking the Upload button in the Authorea Data window. Go back to Article view and find that the article still renders OK. Success. Check that nothing new has been added to layout.md. So why does deleting and re-uploading jam99.bib resolve the problem?

Add a new LaTeX window. Type into the window

\section{Background}

Text.

The window won't render (after a few minutes of waiting). Go to the Data window and delete bibliography/jam99.bib again. Go back to the Article window and the text in the new window now registers.

A Seemingly Serious Limitation

In conclusion, it looks like having a large, 6.8 MB .bib file present in the Authorea repository breaks its ability to render LaTeX text in the Article view. This seeming limitation is extremely frustrating. I would like to cite papers in Authorea using my large .bib file as is, without having to modify it, break it into smaller pieces, or add all my references to the default biblio.bib file that Authorea creates.

Follow up

When I reload Authorea, the article will not render, even with bibliography/jam99.bib absent. It was rendering when I last closed Authorea. What the heck.

References