Bump version to 2.0.0

Better Unit Tests

.editorconfig

@@ -0,0 +1,10 @@
+# editorconfig.org
+root = true
+
+[*]
+indent_style = space
+indent_size = 4
+end_of_line = lf
+charset = utf-8
+trim_trailing_whitespace = true
+insert_final_newline = true

.travis.yml

@@ -1,4 +1,7 @@
+sudo: false
 language: node_js
 node_js:
   - "0.11"
   - "0.10"
+before_install:
+  - npm install svgexport -g

AUTHORS.md

@@ -7,12 +7,36 @@ Names below are ordered by first contribution.
 Author
 ------
 
-- Samy Pessé <samypesse@gmail.com>
-- Aaron O'Mullan <aaron.omullan@gmail.com>
+- Samy Pessé <samypesse@gmail.com> (@SamyPesse)
+- Aaron O'Mullan <aaron.omullan@gmail.com> (@AaronO)
 
 
 Contributors
 ------------
 
-- Nijiko Yonskai
-- Herman Starikov
+- Nijiko Yonskai (@Nijikokun)
+- Herman Starikov (@Hermanya)
+
+Translators
+------------
+
+- German
+	- Winnie (@winniehell)
+- Italian
+	- Giulio Bonanome (@gbonanome)
+- Russian
+	- Andrey Akinshin (@AndreyAkinshin)
+- Norwegian
+	- Knut Melvær (@kmelve)
+- Persian (Farsi)
+	- Mohammad Hossein Mojtahedi (@mhm5000)
+- Portuguese
+	- Ryan O'Mullan (@RyanOM)
+- Spanish
+	- Ryan O'Mullan (@RyanOM)
+- Simplifiled Chinese
+	- Hu Hao (@howiehu)
+- Traditional Chinese
+	- Hu Hao (@howiehu)
+- French
+	- Samy Pessé (@SamyPesse)

CHANGES.md

@@ -1,4 +1,88 @@
 # Release notes
+All notable changes to this project will be documented in this file.
+This project adheres to [Semantic Versioning](http://semver.org/).
+
+## 2.0.0
+- Fix page title of introduction
+- Ignore codeblocks when replacing glossary terms
+- Improve Unit Tests
+- Fix scrolling position in website when preparing page
+
+## 2.0.0-beta.5
+- Fix progress order in json format
+
+## 2.0.0-beta.4
+- Fix default generator for use programmatically
+- Add option "author" for html meta tags
+- Fix links normalization (content and hash)
+
+## 2.0.0-beta.3
+- Fix odd cases with code blocks escaping
+
+## 2.0.0-beta.2
+- Fix definition of entry point title using SUMMARY.md
+
+## 2.0.0-beta.1
+- Fix windows incompatibility
+- Add support for rtl (enabled by default for `ar` and `fa`)
+- Escape code blocks in markdown parser
+- Add Persian/Farsi translation (`fa`)
+- Add Arabic translation (`ar`)
+- Add Bengali translation (`bn`)
+- Provide generator name in template context
+
+## 2.0.0-alpha.9
+- Fix links in sidebar
+- Fix normalization of html link (README to index)
+- Fix html snippets escaping
+
+## 2.0.0-alpha.8
+- Improve locale detection for i18n
+- Fix chapter name for Glossary in pdf
+- Don't escape html in glossary items
+- Fix generation of multilingual book as ebook
+- Add "post" block attribute to post-process
+
+## 2.0.0-alpha.7
+- Fix display of glossary in ebook formats
+- Add default footer and header to pdf
+- Fix generation of json format compatible with 1.x.x
+- Add Simplifiled Chinese and Traditional Chinese translations
+
+## 2.0.0-alpha.6
+- Add es and pt translations
+- Fix replacement of glossary terms
+
+## 2.0.0-alpha.5
+- Fix copy of files/covers
+- Add back `finish:before` hook
+
+## 2.0.0-alpha.4
+- Fix copy of cover for multilingal books
+
+## 2.0.0-alpha.3
+- Norwegian translation
+- Load plugins from book in priority
+
+## 2.0.0-alpha.3
+- Fix init command
+- Update parsers to fix spaces in summary (`gitbook-parsers@0.3.1`)
+
+## 2.0.0-alpha.1
+- Externalize parsing into `gitbook-parsers` module
+- Supports AsciiDoc and reStructuredText
+- Hooks for page (`page:*`) are now deprecated, plugins should extend filters and blocks instead
+- Hooks `summary` and `glossary` (after and before) have been removed
+- Exercises and Quizzes are no longer parsed in the markdown parser
+- Support for more markdown extensions: `.markdown`, `.mdown`
+- Templates are rendered with nunjucks instead of swig, syntax is almost compatible, there is some changes with contexts and filters. `{{ super() }}` should be use instead of `{% parent %}`
+- Clean output folder on build without removing `.git` and `.svn`
+- MathJAX is no longer a default plugin
+- SVG images are converted to PNG during generation of ebooks
+- i18n in website and ebook (ru, it, de, fr)
+- New templating syntax
+- Content references (both internal and external)
+- Glossary terms are handled during generation (also in ebook format)
 
 ## 1.5.0
 - Fix `serve` command, broken by `1.4.2`

Gruntfile.js

@@ -6,8 +6,6 @@ module.exports = function (grunt) {
     grunt.loadNpmTasks('grunt-contrib-less');
     grunt.loadNpmTasks('grunt-contrib-requirejs');
     grunt.loadNpmTasks("grunt-bower-install-simple");
-    grunt.loadNpmTasks('grunt-browserify');
-    grunt.loadNpmTasks('grunt-contrib-uglify');
 
     // Init GRUNT configuraton
     grunt.initConfig({
@@ -79,30 +77,6 @@ module.exports = function (grunt) {
                     }
                 ]
             }
-        },
-        browserify: {
-            dist: {
-                files: {
-                    'gitbook.js': [
-                        './lib/parse/index.js'
-                    ],
-                },
-                options: {
-                    postBundleCB: function (err, src, next) {
-                        return next(null, '(function () { var define = undefined; '+src+'; })();')
-                    },
-                    browserifyOptions: {
-                        'standalone': "gitbook"
-                    }
-                }
-            }
-        },
-        uglify: {
-            dist: {
-                files: {
-                    'gitbook.min.js': ['gitbook.js']
-                }
-            }
         }
     });
 
@@ -116,14 +90,7 @@ module.exports = function (grunt) {
         'copy:vendors'
     ]);
 
-    // Bundle the library
-    grunt.registerTask('bundle', [
-        'browserify',
-        'uglify'
-    ]);
-
     grunt.registerTask('default', [
-        'build',
-        'bundle'
+        'build'
     ]);
-};
+};

README.md

@@ -1,10 +1,17 @@
 GitBook
 =======
 
-[![Build Status](https://travis-ci.org/GitbookIO/gitbook.png?branch=master)](https://travis-ci.org/GitbookIO/gitbook)
 [![NPM version](https://badge.fury.io/js/gitbook.svg)](http://badge.fury.io/js/gitbook)
+[![Build Status](https://travis-ci.org/GitbookIO/gitbook.png?branch=master)](https://travis-ci.org/GitbookIO/gitbook)
+[![Windows Build Status](https://ci.appveyor.com/api/projects/status/github/GitbookIO/gitbook?branch=master&svg=true)](https://ci.appveyor.com/project/SamyPesse/gitbook)
 
-GitBook is a command line tool (and Node.js library) for building beautiful books using GitHub/Git and Markdown. Here is an example: [Learn Javascript](https://www.gitbook.io/book/GitBookIO/javascript). You can publish book easily online using [gitbook.io](https://www.gitbook.io) and an [editor](https://github.com/GitbookIO/editor) is available for Windows, Mac and Linux. You can follow [@GitBookIO](https://twitter.com/GitBookIO) on Twitter. Complete documentation is available at [help.gitbook.io](http://help.gitbook.io/).
+GitBook is a command line tool (and Node.js library) for building beautiful books using GitHub/Git and Markdown (or AsciiDoc). Here is an example: [Learn Javascript](https://www.gitbook.com/book/GitBookIO/javascript).
+
+You can publish and host book easily online using [gitbook.com](https://www.gitbook.com), a web-editor is [also available](https://www.gitbook.com/editor).
+
+Stay updated by following [@GitBookIO](https://twitter.com/GitBookIO) on Twitter or [GitBook](https://www.facebook.com/gitbookcom) on Facebook.
+
+Complete documentation is available at [help.gitbook.com](http://help.gitbook.com/).
 
 ![Image](https://raw.github.com/GitbookIO/gitbook/master/preview.png)
 
@@ -13,10 +20,11 @@ GitBook is a command line tool (and Node.js library) for building beautiful book
 GitBook can be installed from **NPM** using:
 
 ```
-$ npm install gitbook -g
+$ npm install gitbook-cli -g
 ```
 
-Create the directories and files for a book from its [SUMMARY.md](https://github.com/GitbookIO/gitbook#book-format) file using 
+Create the directories and files for a book from its [SUMMARY.md](https://github.com/GitbookIO/gitbook#book-format) file (if existing) using
+
 ```
 $ gitbook init
 ```
@@ -24,125 +32,40 @@ $ gitbook init
 You can serve a repository as a book using:
 
 ```
-$ gitbook serve ./repository
+$ gitbook serve
 ```
 
 Or simply build the static website using:
 
 ```
-$ gitbook build ./repository --output=./outputFolder
+$ gitbook build
 ```
 
-Options for commands `build` and `serve` are:
+## Features
 
-```
--o, --output <directory>  Path to output directory, defaults to ./_book
--f, --format <name>       Change generation format, defaults to site, availables are: site, page, ebook, json
---config <config file>    Configuration file to use, defaults to book.js or book.json
-```
-
-GitBook loads the default configuration from a `book.json` file in the repository if it exists.
-
-Here are the options that can be stored in this file:
-
-```js
-{
-    // Folders to use for output
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "output": null,
-
-    // Generator to use for building
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "generator": "site",
-
-    // Book metadats (somes are extracted from the README by default)
-    "title": null,
-    "description": null,
-    "isbn": null,
-
-    // For ebook format, the extension to use for generation (default is detected from output extension)
-    // "epub", "pdf", "mobi"
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "extension": null,
-
-    // Plugins list, can contain "-name" for removing default plugins
-    "plugins": [],
-
-    // Global configuration for plugins
-    "pluginsConfig": {
-        "fontSettings": {
-            "theme": "sepia", "night" or "white",
-            "family": "serif" or "sans",
-            "size": 1 to 4
-        }
-    },
-
-    // Variables for templating
-    "variables": {},
-
-    // Links in template (null: default, false: remove, string: new value)
-    "links": {
-    	// Custom links at top of sidebar
-    	"sidebar": {
-    	    "Custom link name": "https://customlink.com"
-    	},
-
-        // Sharing links
-        "sharing": {
-            "google": null,
-            "facebook": null,
-            "twitter": null,
-            "weibo": null,
-            "all": null
-        }
-    },
-
-
-    // Options for PDF generation
-    "pdf": {
-        // Add page numbers to the bottom of every page
-        "pageNumbers": false,
-
-        // Font size for the fiel content
-        "fontSize": 12,
-
-        // Paper size for the pdf
-        // Choices are [u’a0’, u’a1’, u’a2’, u’a3’, u’a4’, u’a5’, u’a6’, u’b0’, u’b1’, u’b2’, u’b3’, u’b4’, u’b5’, u’b6’, u’legal’, u’letter’]
-        "paperSize": "a4",
-
-        // Margin (in pts)
-        // Note: 72 pts equals 1 inch
-        "margin": {
-            "right": 62,
-            "left": 62,
-            "top": 36,
-            "bottom": 36
-        },
-
-        //Header HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
-        "headerTemplate": null,
-
-        //Footer HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
-        "footerTemplate": null
-    }
-}
-```
-
-You can publish your books to our index by visiting [GitBook.io](http://www.gitbook.io)
+* [Output as a website or ebook (pdf, epub, mobi)](http://help.gitbook.com/format/output.html)
+* [Multi-Languages](http://help.gitbook.com/format/languages.html)
+* [Glossary](http://help.gitbook.com/format/glossary.html)
+* [Cover](http://help.gitbook.com/format/cover.html)
+* [AsciiDoc Support](http://help.gitbook.com/format/asciidoc.html)
+* [Variables and Templating](http://help.gitbook.com/format/templating.html)
+* [Content References](http://help.gitbook.com/format/conrefs.html)
+* [Plugins](http://help.gitbook.com/plugins/index.html)
+* Interractive reader website:
+    * Search
+    * Font Settings (Serif, Sans Serif)
+    * Themes: white, sepia, night
 
 ## Output Formats
 
 GitBook can generate your book in the following formats:
 
 * **Static Website**: This is the default format. It generates a complete interactive static website that can be, for example, hosted on GitHub Pages.
-* **eBook**: A complete eBook with exercise solutions at the end of the book.  You need to have [ebook-convert](http://manual.calibre-ebook.com/cli/ebook-convert.html) installed.  You can specify the eBook filename with the `-o` option, otherwise `book` will be used.
-  * Generate a **PDF** using:  `gitbook pdf ./myrepo`
-  * Generate a **ePub** using: `gitbook epub ./myrepo`
-  * Generate a **MOBI** using: `gitbook mobi ./myrepo`
-* **JSON**: This format is used for debugging or extracting metadata from a book. Generate this format using: ```gitbook build ./myrepo -f json```.
+* **eBook**: You need to have [ebook-convert](http://manual.calibre-ebook.com/cli/ebook-convert.html) installed.  You can specify the eBook filename as the second argument, otherwise `book` will be used.
+  * Generate a **PDF** using:  `gitbook pdf ./myrepo ./mybook.pdf`
+  * Generate a **ePub** using: `gitbook epub ./myrepo ./mybook.epub`
+  * Generate a **MOBI** using: `gitbook mobi ./myrepo ./mybook.mobi`
+* **JSON**: This format is used for debugging or extracting metadata from a book. Generate this format using: ```gitbook build ./myrepo --format=json```.
 
 ## Book Format
 
@@ -196,7 +119,64 @@ Definition for this term
 
 # Another term
 With it's definition, this can contain bold text and all other kinds of inline markup ...
+```
 
+#### Variables and Templating
+
+A set of variables can be defined in the `book.json`:
+
+```js
+{
+    "variables": {
+        "host": "mybook.com"
+    }
+}
+```
+
+These variables can be used in the markdown files:
+
+```
+The host is {{ book.host }}
+```
+
+You can also use condition with these variables:
+
+```
+{% if book.host == "mybook.com" %}
+
+{% else %}
+
+{% endif %}
+```
+
+Variables of `book.json` are available in the `book` namespace. You can also access informations about the `file` itself and the `gitbook` version:
+
+```
+My file is {{ file.path }}
+Modified at {{ file.mtime }}
+Book built with GitBook {{ gitbook.version }}
+```
+
+#### Content References
+
+You can use "content references," or conrefs, when writing books or documentation using GitBook.
+
+Include a file from the same book:
+
+```
+{% include "./test.md" %}
+```
+
+or from a git repository (with a specific revision):
+
+```
+{% include "git+https://github.com/GitbookIO/documentation.git/README.md#1.0.1" %}
+```
+
+Includes can be used with variables (see [Variables and Templating](#variables-and-templating)):
+
+```
+{% include book.ref_doc_readme %}
 ```
 
 #### Ignoring files & folders
@@ -212,49 +192,38 @@ The best resolution is **1800x2360**. The generation of the cover can be done au
 
 A small version of the cover can also be set by creating a file: **/cover_small.jpg**.
 
-#### Publish your book
+#### AsciiDoc
 
-The platform [GitBook.io](https://www.gitbook.io/) is like an "Heroku for books": you can create a book on it (public, paid, or private) and update it using **git push**.
+Since version 2.0.0, AsciiDoc can be used instead of Markdown, simply replace the `.md` by the `.adoc` extension. Chapters in the summary are detected from an ordered list in the `SUMMARY.adoc`.
 
-#### Plugins
+## Publish your book
+
+The platform [GitBook.com](https://www.gitbook.com/) is like an "Heroku for books": you can create a book on it (public, paid, or private) and update it using **git push**.
+
+## Plugins
 
 Plugins can be used to extend your book's functionality. Read [GitbookIO/plugin](https://github.com/GitbookIO/plugin) for more information about how to build a plugin for GitBook.
 
-Plugins needed to build a book can be installed using: `gitbook install ./`.
+Plugins needed to build a book can be installed using: `gitbook install ./`. You can find plugins at [plugins.gitbook.com](http://plugins.gitbook.com).
 
-##### Official plugins:
 
-| Name | Description |
-| ----- | ---- |
-| [exercises](https://github.com/GitbookIO/plugin-exercises) | Add interactive exercises to your book. |
-| [quizzes](https://github.com/GitbookIO/plugin-quizzes) | Add interactive quizzes to your book. |
-| [mathjax](https://github.com/GitbookIO/plugin-mathjax) | Displays mathematical notation in the book. |
-| [mixpanel](https://github.com/GitbookIO/plugin-mixpanel) | Mixpanel tracking for your book |
-| [infinitescroll](https://github.com/GitbookIO/gitbook-plugin-infinitescroll) | Infinite Scrolling |
+## Debugging
 
-##### Other plugins:
-
-| Name | Description |
-| ----- | ---- |
-| [Google Analytics](https://github.com/GitbookIO/plugin-ga) | Google Analytics tracking for your book |
-| [Disqus](https://github.com/GitbookIO/plugin-disqus) | Disqus comments integration in your book |
-| [Autocover](https://github.com/GitbookIO/plugin-autocover) | Generate a cover for your book |
-| [Transform annoted quotes to notes](https://github.com/erixtekila/gitbook-plugin-richquotes) | Allow extra markdown markup to render blockquotes as nice notes |
-| [Send code to console](https://github.com/erixtekila/gitbook-plugin-toconsole) | Evaluate javascript block in the browser inspector's console |
-| [Revealable sections](https://github.com/mrpotes/gitbook-plugin-reveal) | Reveal sections of the page using buttons made from the first title in each section |
-| [Markdown within HTML](https://github.com/mrpotes/gitbook-plugin-nestedmd) | Process markdown within HTML blocks - allows custom layout options for individual pages |
-| [Bootstrap JavaScript plugins](https://github.com/mrpotes/gitbook-plugin-bootstrapjs) | Use the [Bootstrap JavaScript plugins](http://getbootstrap.com/javascript) in your online GitBook |
-| [Piwik Open Analytics](https://github.com/emmanuel-keller/gitbook-plugin-piwik) | Piwik Open Analytics tracking for your book |
-| [Heading Anchors](https://github.com/rlmv/gitbook-plugin-anchors) | Add linkable Github-style anchors to headings |
-| [JSBin](https://github.com/jcouyang/gitbook-plugin-jsbin) | Embedded jsbin frame into your book |
-| [gitbook-grvis](https://github.com/romanlytkin/gitbook-grvis) | Gitbook GrViz plugin is used to select from markdown dot and converting it into a picture format svg |
-| [gitbook-plantuml](https://github.com/romanlytkin/gitbook-plantuml) | Gitbook PlantUml plugin is used to select from markdown uml and converting it into a picture format svg |
-
-#### Debugging
-
-You can use the environment variable `DEBUG=true` to get better error messages (with stack trace). For example:
+You can use the options `--log=debug` and `--debug` to get better error messages (with stack trace). For example:
 
 ```
-$ export DEBUG=true
-$ gitbook build ./
+$ gitbook build ./ -- log=debug --debug
 ```
+
+#### How to use the latest commit from GitBook in gitbook-cli
+
+To use the latest commit from `GitBook/gitbook` with `gitbook-cli`:
+
+```
+$ git clone https://github.com/GitbookIO/gitbook.git ./gitbook
+$ gitbook versions:link 2.0.0 ./gitbook
+```
+
+Now the version tag `2.0.0` wil be associated with the `./gitbook` folder.
+
+You can uninstall it using: `gitbook versions:uninstall 2.0.0`.

appveyor.yml

@@ -0,0 +1,29 @@
+# Fix line endings in Windows. (runs before repo cloning)
+init:
+  - git config --global core.autocrlf input
+
+# Test against these versions of Node.js.
+environment:
+  matrix:
+    - nodejs_version: "0.10"
+    - nodejs_version: "0.11"
+
+# Install scripts. (runs after repo cloning)
+install:
+  # Get the latest stable version of Node.js or io.js
+  - ps: Install-Product node $env:nodejs_version
+  # install svgexport
+  - npm install svgexport -g
+  # install modules
+  - npm install
+
+# Post-install test scripts.
+test_script:
+  # Output useful info for debugging.
+  - node --version
+  - npm --version
+  # run tests
+  - npm test
+
+# Don't actually build.
+build: off

bin/build.js

@@ -1,56 +0,0 @@
-var path = require('path');
-var Q = require('q');
-var _ = require('lodash');
-var fs = require('fs');
-
-var utils = require('./utils');
-var generate = require("../lib/generate");
-var parse = require("../lib/parse");
-var generators = require("../lib/generate").generators;
-
-var buildCommand = function(command) {
-    return command
-    .option('-v, --verbose', 'Activate verbose mode, useful for debugging errors')
-    .option('-o, --output <directory>', 'Path to output directory, defaults to ./_book')
-    .option('-f, --format <name>', 'Change generation format, defaults to site, availables are: '+_.keys(generators).join(", "))
-    .option('--config <config file>', 'Configuration file to use, defaults to book.js or book.json');
-};
-
-
-var buildEbookCommand = function(command) {
-    return buildCommand(command)
-    .option('-c, --cover <path>', 'Cover image, default is cover.jpg if exists');
-};
-
-var makeBuildFunc = function(converter) {
-    return  function(dir, options) {
-        dir = dir || process.cwd();
-        outputDir = options.output;
-
-        // Set debugging
-        if(options.verbose) {
-            process.env.DEBUG = "true";
-        }
-
-        console.log('Starting build ...');
-        return converter(
-            _.extend({}, options || {}, {
-                input: dir,
-                output: outputDir,
-                generator: options.format,
-                configFile: options.config
-            })
-        )
-        .then(function(output) {
-            console.log("Successfully built!");
-            return output;
-        });
-    };
-};
-
-module.exports = {
-    folder: makeBuildFunc(generate.folder),
-    file: makeBuildFunc(generate.file),
-    command: buildCommand,
-    commandEbook: buildEbookCommand
-};

bin/gitbook.js

@@ -1,159 +1,7 @@
 #! /usr/bin/env node
 
-var Q = require('q');
-var _ = require('lodash');
-var path = require('path');
-var prog = require('commander');
-var tinylr = require('tiny-lr-fork');
+var color = require('bash-color');
 
-var pkg = require('../package.json');
-var genbook = require("../lib/generate");
-var initDir = require("../lib/generate/init");
-var fs = require('../lib/generate/fs');
-
-var utils = require('./utils');
-var action = utils.action;
-var build = require('./build');
-var Server = require('./server');
-var platform = require("./platform");
-
-// General options
-prog
-.version(pkg.version);
-
-build.command(prog.command('build [source_dir]'))
-.description('Build a gitbook from a directory')
-.action(action(build.folder));
-
-build.command(prog.command('serve [source_dir]'))
-.description('Build then serve a gitbook from a directory')
-.option('-p, --port <port>', 'Port for server to listen on', 4000)
-.option('--no-watch', 'Disable restart with file watching')
-.action(action(function(dir, options) {
-    var server = new Server();
-
-    // init livereload server
-    var lrOptions = {port: 35729};
-    var lrServer = tinylr(lrOptions);
-    var lrPath = undefined;
-    lrServer.listen(lrOptions.port, function(err) {
-      if (err) { return console.log(err); }
-      console.log('Live reload server started on port: ' + lrOptions.port);
-    });
-
-    var generate = function() {
-        if (server.isRunning()) {
-            console.log("Stopping server");
-        }
-
-        return server.stop()
-        .then(function() {
-            return build.folder(dir, _.extend(options || {}, {
-                defaultsPlugins: ["livereload"]
-            }));
-        })
-        .then(function(_options) {
-            console.log();
-            console.log('Starting server ...');
-            return server.start(_options.output, options.port)
-            .then(function() {
-                console.log('Serving book on http://localhost:'+options.port);
-
-                if (lrPath) {
-                  // trigger livereload
-                  lrServer.changed({body:{files:[lrPath]}})
-                }
-
-                if (!options.watch) return;
-                return utils.watch(_options.input)
-                .then(function(filepath) {
-                    // set livereload path
-                    lrPath = filepath;
-                    console.log("Restart after change in files");
-                    console.log('');
-                    return generate();
-                })
-            })
-        })
-        .fail(utils.logError);
-    };
-
-    console.log('Press CTRL+C to quit ...');
-    console.log('')
-
-    return generate();
-}));
-
-build.commandEbook(prog.command('install [source_dir]'))
-.description('Install plugins for a book')
-.action(action(function(dir, options) {
-    dir = dir || process.cwd();
-
-    console.log("Install plugins in", dir);
-
-    return genbook.config.read({
-        input: dir
-    })
-    .then(function(options) {
-        return genbook.Plugin.install(options);
-    })
-    .then(function() {
-        console.log("Successfully installed plugins!");
-    });
-}));
-
-build.commandEbook(prog.command('pdf [source_dir]'))
-.description('Build a gitbook as a PDF')
-.action(action(function(dir, options) {
-    return build.file(dir, _.extend(options, {
-        extension: "pdf",
-        format: "ebook"
-    }));
-}));
-
-build.commandEbook(prog.command('epub [source_dir]'))
-.description('Build a gitbook as a ePub book')
-.action(action(function(dir, options) {
-    return build.file(dir, _.extend(options, {
-        extension: "epub",
-        format: "ebook"
-    }));
-}));
-
-build.commandEbook(prog.command('mobi [source_dir]'))
-.description('Build a gitbook as a Mobi book')
-.action(action(function(dir, options) {
-    return build.file(dir, _.extend(options, {
-        extension: "mobi",
-        format: "ebook"
-    }));
-}));
-
-prog
-.command('init [source_dir]')
-.description('Create files and folders based on contents of SUMMARY.md')
-.action(action(function(dir) {
-    dir = dir || process.cwd();
-    return initDir(dir);
-}));
-
-prog
-.command('publish [source_dir]')
-.description('Publish content to the associated gitbook.io book')
-.action(action(function(dir) {
-    dir = dir || process.cwd();
-    return platform.publish(dir);
-}));
-
-prog
-.command('git:remote [source_dir] [book_id]')
-.description('Adds a git remote to a book repository')
-.action(action(function(dir, bookId) {
-    dir = dir || process.cwd();
-    return platform.remote(dir, bookId);
-}));
-
-// Parse and fallback to help if no args
-if(_.isEmpty(prog.parse(process.argv).args) && process.argv.length === 2) {
-    prog.help();
-}
+console.log(color.red("You need to install 'gitbook-cli' to have access to the gitbook command anywhere on your system."));
+console.log(color.red("If you've installed this package globally, you need to uninstall it."));
+console.log(color.red(">> Run 'npm uninstall -g gitbook' then 'npm install -g gitbook-cli'"));

bin/platform.js

@@ -1,53 +0,0 @@
-var Q = require("q");
-var utils = require("./utils");
-
-var publish = function(folder) {
-    if (!folder) {
-        console.log("Need a repository folder");
-        return process.exit(-1);
-    }
-
-    utils.gitCmd("push", ["gitbook", "master"])
-    .then(function(out) {
-        console.log(out.stdout);
-    }, function(err) {
-        if (err.code == 128) {
-            console.log("No book on gitbook.io is configured with this git repository.");
-            console.log("Run 'gitbook git:remote username/book' to intialize this repository.");
-        } else {
-            console.log(err.message);
-        }
-        process.exit(-1);
-    });
-};
-
-var remote = function(folder, bookId) {
-    if (!folder || !bookId) {
-        console.log("Need a repository folder and a book id");
-        return process.exit(-1);
-    }
-
-    var url = "https://push.gitbook.io/"+bookId+".git";
-    var addRemote = function() {
-        return utils.gitCmd("remote", ["add", "gitbook", url]);
-    }
-
-    addRemote()
-    .fail(function(err) {
-        if (err.code == 128) {
-            return utils.gitCmd("remote", ["rm", "gitbook"]).then(addRemote);
-        }
-        return Q.reject(err);
-    })
-    .then(function(out) {
-        console.log("Book remote '"+url+"' added to the folder");
-    }, function(err) {
-        console.log(err.message);
-        process.exit(-1);
-    });
-};
-
-module.exports = {
-    publish: publish,
-    remote: remote
-};

bin/utils.js

@@ -1,96 +0,0 @@
-var Q = require('q');
-var _ = require('lodash');
-
-var exec = require('child_process').exec;
-var http = require('http');
-var send = require('send');
-
-var path = require('path');
-var Gaze = require('gaze').Gaze;
-
-function watch(dir) {
-    var d = Q.defer();
-    dir = path.resolve(dir);
-
-    var gaze = new Gaze("**/*.md", {
-        cwd: dir
-    });
-
-    gaze.once("all", function(e, filepath) {
-        gaze.close();
-
-        d.resolve(filepath);
-    });
-    gaze.once("error", function(err) {
-        gaze.close();
-
-        d.reject(err);
-    });
-
-    return d.promise;
-}
-
-// exit wraps a promise
-// and forcefully quits the program when the promise is resolved
-function exit(promise) {
-    promise
-    .then(function() {
-        // Prevent badly behaving plugins
-        // from making the process hang
-        process.exit(0);
-    }, function(err) {
-        // Log error
-        logError(err);
-
-        // Exit process with failure code
-        process.exit(-1);
-    });
-}
-
-// CLI action wrapper, calling exit when finished
-function action(f) {
-    return function() {
-        // Call func and get optional promise
-        var p = f.apply(null, arguments);
-
-        // Exit process
-        return exit(Q(p));
-    }
-}
-
-function logError(err) {
-    var message = err.message || err;
-    if (process.env.DEBUG != null) message = err.stack || message;
-    console.log(message);
-    return Q.reject(err);
-};
-
-function runGitCommand(command, args) {
-    var d = Q.defer(), child;
-    args = ["git", command].concat(args).join(" ");
-
-    child = exec(args, function (error, stdout, stderr) {
-        if (error !== null) {
-            error.stdout = stdout;
-            error.stderr = stderr;
-            d.reject(error);
-        } else {
-            d.resolve({
-                stdout: stdout,
-                stderr: stderr
-            })
-        }
-    });
-
-    return d.promise;
-};
-
-
-// Exports
-module.exports = {
-    exit: exit,
-    action: action,
-    watch: watch,
-    logError: logError,
-    gitCmd: runGitCommand
-};

lib/book.js

@@ -0,0 +1,796 @@
+var Q = require("q");
+var _ = require("lodash");
+var path = require("path");
+var util = require("util");
+var lunr = require('lunr');
+var parsers = require("gitbook-parsers");
+var color = require('bash-color');
+
+var fs = require("./utils/fs");
+var parseNavigation = require("./utils/navigation");
+var parseProgress = require("./utils/progress");
+var pageUtil = require("./utils/page");
+var batch = require("./utils/batch");
+var links = require("./utils/links");
+var logger = require("./utils/logger");
+
+var Configuration = require("./configuration");
+var TemplateEngine = require("./template");
+var Plugin = require("./plugin");
+var PluginsList = require("./pluginslist");
+
+var generators = require("./generators");
+
+var Book = function(root, context, parent) {
+    var that = this;
+
+    this.context = _.defaults(context || {}, {
+        // Extend book configuration
+        config: {},
+
+        // Log function
+        log: function(msg) {
+            process.stdout.write(msg);
+        },
+
+        // Log level
+        logLevel: "info"
+    });
+
+    // Log
+    this.log = logger(this.context.log, this.context.logLevel);
+
+    // Root folder of the book
+    this.root = path.resolve(root);
+
+    // Parent book
+    this.parent = parent;
+
+    // Configuration
+    this.config = new Configuration(this, this.context.config);
+    Object.defineProperty(this, "options", {
+        get: function () {
+            return this.config.options;
+        }
+    });
+
+    // Template
+    this.template = new TemplateEngine(this);
+
+    // Summary
+    this.summary = {};
+    this.navigation = [];
+
+    // Glossary
+    this.glossary = [];
+
+    // Langs
+    this.langs = [];
+
+    // Sub-books
+    this.books = [];
+
+    // Files in the book
+    this.files = [];
+
+    // List of plugins
+    this.plugins = new PluginsList(this);
+
+    // Structure files
+    this.summaryFile = null;
+    this.glossaryFile = null;
+    this.readmeFile = null;
+    this.langsFile = null;
+
+    // Search Index
+    this.searchIndex = lunr(function () {
+        this.ref('url');
+
+        this.field('title', { boost: 10 });
+        this.field('body');
+    });
+
+    // Bind methods
+    _.bindAll(this);
+};
+
+// Return string representation
+Book.prototype.toString = function() {
+    return "[Book "+this.root+"]";
+};
+
+// Initialize and parse the book: config, summary, glossary
+Book.prototype.parse = function() {
+    var that = this;
+    var multilingual = false;
+
+    return this.parseConfig()
+
+    .then(function() {
+        return that.parsePlugins();
+    })
+
+    .then(function() {
+        return that.parseLangs()
+        .then(function() {
+            multilingual = that.langs.length > 0;
+            if (multilingual) that.log.info.ln("Parsing multilingual book, with", that.langs.length, "lanuages");
+
+            // Sub-books that inherit from the current book configuration
+            that.books = _.map(that.langs, function(lang) {
+                that.log.info.ln("Preparing language book", lang.lang);
+                return new Book(
+                    path.join(that.root, lang.path),
+                    _.merge({}, that.context, {
+                        config: _.extend({}, that.options, {
+                            'output': path.join(that.options.output, lang.lang),
+                            'language': lang.lang
+                        })
+                    }),
+                    that
+                )
+            });
+        });
+    })
+
+    .then(function() {
+        if (multilingual) return;
+        return that.listAllFiles();
+    })
+    .then(function() {
+        if (multilingual) return;
+        return that.parseReadme();
+    })
+    .then(function() {
+        if (multilingual) return;
+        return that.parseSummary();
+    })
+    .then(function() {
+        if (multilingual) return;
+        return that.parseGlossary();
+    })
+
+    .then(function() {
+        // Init sub-books
+        return _.reduce(that.books, function(prev, book) {
+            return prev.then(function() {
+                return book.parse();
+            });
+        }, Q());
+    })
+
+    .thenResolve(this);
+};
+
+// Generate the output
+Book.prototype.generate = function(generator) {
+    var that = this;
+    that.options.generator = generator || that.options.generator;
+
+    that.log.info.ln("start generation with", that.options.generator, "generator");
+    return Q()
+
+    // Clean output folder
+    .then(function() {
+        that.log.info("clean", that.options.generator, "generator");
+        return fs.clean(that.options.output)
+        .progress(function(p) {
+            that.log.debug.ln("remove", p.file, "("+p.i+"/"+p.count+")");
+        })
+        .then(function() {
+            that.log.info.ok();
+        });
+    })
+
+    // Create generator
+    .then(function() {
+        var Generator = generators[generator];
+        if (!Generator) throw "Generator '"+that.options.generator+"' doesn't exist";
+        generator = new Generator(that);
+
+        return generator.prepare();
+    })
+
+    // Generate content
+    .then(function() {
+        if (that.isMultilingual()) {
+            return that.generateMultiLingual(generator);
+        } else {
+            // Separate list of files into the different operations needed
+            var ops = _.groupBy(that.files, function(file) {
+                if (file[file.length -1] == "/") {
+                    return "directories";
+                } else if (_.contains(parsers.extensions, path.extname(file)) && that.navigation[file]) {
+                    return "content";
+                } else {
+                    return "files";
+                }
+            });
+
+
+            return Q()
+
+            // First, let's create folder
+            .then(function() {
+                return _.reduce(ops["directories"] || [], function(prev, folder) {
+                    return prev.then(function() {
+                        that.log.debug.ln("transferring folder", folder);
+                        return Q(generator.transferFolder(folder));
+                    });
+                }, Q());
+            })
+
+            // Then, let's copy other files
+            .then(function() {
+                return Q.all(_.map(ops["files"] || [], function(file) {
+                    that.log.debug.ln("transferring file", file);
+                    return Q(generator.transferFile(file));
+                }));
+            })
+
+            // Finally let's generate content
+            .then(function() {
+                var nFiles = (ops["content"] || []).length;
+                return _.reduce(ops["content"] || [], function(prev, file, i) {
+                    return prev.then(function() {
+                        var p = ((i*100)/nFiles).toFixed(0)+"%";
+                        that.log.info.ln("processing", file, p);
+                        return Q(generator.convertFile(file));
+                    });
+                }, Q());
+            });
+        }
+    })
+
+    // Finish generation
+    .then(function() {
+        return generator.callHook("finish:before");
+    })
+    .then(function() {
+        return generator.finish();
+    })
+    .then(function() {
+        return generator.callHook("finish");
+    })
+    .then(function() {
+        that.log.info.ln("generation is finished");
+    });
+};
+
+// Generate the output for a multilingual book
+Book.prototype.generateMultiLingual = function(generator) {
+    var that = this;
+
+    return Q()
+    .then(function() {
+        // Generate sub-books
+        return _.reduce(that.books, function(prev, book) {
+            return prev.then(function() {
+                return book.generate(that.options.generator);
+            });
+        }, Q());
+    });
+};
+
+// Extract files from ebook generated
+Book.prototype.generateFile = function(output, options) {
+    var book = this;
+
+    options = _.defaults(options || {}, {
+        ebookFormat: path.extname(output).slice(1)
+    });
+    output = output || path.resolve(book.root, "book."+options.ebookFormat);
+
+    return fs.tmp.dir()
+    .then(function(tmpDir) {
+        book.setOutput(tmpDir);
+
+        return book.generate(options.ebookFormat)
+        .then(function(_options) {
+            var copyFile = function(lang) {
+                var _outputFile = output;
+                var _tmpDir = tmpDir;
+
+                if (lang) {
+                    _outputFile = _outputFile.slice(0, -path.extname(_outputFile).length)+"_"+lang+path.extname(_outputFile);
+                    _tmpDir = path.join(_tmpDir, lang);
+                }
+
+                book.log.debug.ln("copy ebook to", _outputFile);
+                return fs.copy(
+                    path.join(_tmpDir, "index."+options.ebookFormat),
+                    _outputFile
+                );
+            };
+
+            // Multi-langs book
+            return Q()
+            .then(function() {
+                if (book.isMultilingual()) {
+                    return Q.all(
+                        _.map(book.langs, function(lang) {
+                            return copyFile(lang.lang);
+                        })
+                    )
+                    .thenResolve(book.langs.length);
+                } else {
+                    return copyFile().thenResolve(1);
+                }
+            })
+            .then(function(n) {
+                book.log.info.ok(n+" file(s) generated");
+
+                return fs.remove(tmpDir);
+            });
+        });
+    });
+};
+
+// Parse configuration
+Book.prototype.parseConfig = function() {
+    var that = this;
+
+    that.log.info("loading book configuration....")
+    return that.config.load()
+    .then(function() {
+        that.log.info.ok();
+    });
+};
+
+// Parse list of plugins
+Book.prototype.parsePlugins = function() {
+    var that = this;
+    var failed = [];
+
+    // Load plugins
+    return that.plugins.load(that.options.plugins)
+    .then(function() {
+        if (_.size(that.plugins.failed) > 0) return Q.reject(new Error("Error loading plugins: "+that.plugins.failed.join(",")+". Run 'gitbook install' to install plugins from NPM."));
+
+        that.log.info.ok(that.plugins.count()+" plugins loaded");
+        that.log.debug.ln("normalize plugins list");
+    });
+};
+
+// Parse readme to extract defaults title and description
+Book.prototype.parseReadme = function() {
+    var that = this;
+    var structure = that.config.getStructure("readme");
+    that.log.debug.ln("start parsing readme:", structure);
+
+    return that.findFile(structure)
+    .then(function(readme) {
+        if (!readme) throw "No README file";
+        if (!_.contains(that.files, readme.path)) throw "README file is ignored";
+
+        that.readmeFile = readme.path;
+        that._defaultsStructure(that.readmeFile);
+
+        that.log.debug.ln("readme located at", that.readmeFile);
+        return that.template.renderFile(that.readmeFile)
+        .then(function(content) {
+            return readme.parser.readme(content);
+        });
+    })
+    .then(function(readme) {
+        that.options.title = that.options.title || readme.title;
+        that.options.description = that.options.description || readme.description;
+    });
+};
+
+
+// Parse langs to extract list of sub-books
+Book.prototype.parseLangs = function() {
+    var that = this;
+
+    var structure = that.config.getStructure("langs");
+    that.log.debug.ln("start parsing languages index:", structure);
+
+    return that.findFile(structure)
+    .then(function(langs) {
+        if (!langs) return [];
+
+        that.langsFile = langs.path;
+        that._defaultsStructure(that.langsFile);
+
+        that.log.debug.ln("languages index located at", that.langsFile);
+        return that.template.renderFile(that.langsFile)
+        .then(function(content) {
+            return langs.parser.langs(content);
+        });
+    })
+    .then(function(langs) {
+        that.langs = langs;
+    });
+};
+
+// Parse summary to extract list of chapters
+Book.prototype.parseSummary = function() {
+    var that = this;
+
+    var structure = that.config.getStructure("summary");
+    that.log.debug.ln("start parsing summary:", structure);
+
+    return that.findFile(structure)
+    .then(function(summary) {
+        if (!summary) throw "No SUMMARY file";
+
+        // Remove the summary from the list of files to parse
+        that.summaryFile = summary.path;
+        that._defaultsStructure(that.summaryFile);
+        that.files = _.without(that.files, that.summaryFile);
+
+        that.log.debug.ln("summary located at", that.summaryFile);
+        return that.template.renderFile(that.summaryFile)
+        .then(function(content) {
+            return summary.parser.summary(content, {
+                entryPoint: that.readmeFile,
+                entryPointTitle: "",
+                files: that.files
+            });
+        });
+    })
+    .then(function(summary) {
+        that.summary = summary;
+        that.navigation = parseNavigation(that.summary, that.files);
+    });
+};
+
+// Parse glossary to extract terms
+Book.prototype.parseGlossary = function() {
+    var that = this;
+
+    var structure = that.config.getStructure("glossary");
+    that.log.debug.ln("start parsing glossary: ", structure);
+
+    return that.findFile(structure)
+    .then(function(glossary) {
+        if (!glossary) return [];
+
+        // Remove the glossary from the list of files to parse
+        that.glossaryFile = glossary.path;
+        that._defaultsStructure(that.glossaryFile);
+        that.files = _.without(that.files, that.glossaryFile);
+
+        that.log.debug.ln("glossary located at", that.glossaryFile);
+        return that.template.renderFile(that.glossaryFile)
+        .then(function(content) {
+            return glossary.parser.glossary(content);
+        });
+    })
+    .then(function(glossary) {
+        that.glossary = glossary;
+    });
+};
+
+// Parse a page
+Book.prototype.parsePage = function(filename, options) {
+    var that = this, page = {};
+    options = _.defaults(options || {}, {
+        // Transform svg images
+        convertImages: false,
+
+        // Interpolate before templating
+        interpolateTemplate: _.identity,
+
+        // Interpolate after templating
+        interpolateContent: _.identity
+    });
+
+    var interpolate = function(fn) {
+        return Q(fn(page))
+        .then(function(_page) {
+            page = _page;
+        });
+    };
+
+    that.log.debug.ln("start parsing file", filename);
+
+    var extension = path.extname(filename);
+    var filetype = parsers.get(extension);
+
+    if (!filetype) return Q.reject(new Error("Can't parse file: "+filename));
+
+    // Type of parser used
+    page.type = filetype.name;
+
+    // Path relative to book
+    page.path = filename;
+
+    // Path absolute in the system
+    page.rawPath = path.resolve(that.root, filename);
+
+    // Progress in the book
+    page.progress = parseProgress(that.navigation, filename);
+
+    that.log.debug.ln("render template", filename);
+
+    // Read file content
+    return that.readFile(page.path)
+    .then(function(content) {
+        page.content = content;
+
+        return interpolate(options.interpolateTemplate);
+    })
+
+    // Prepare page markup
+    .then(function() {
+        return filetype.page.prepare(page.content)
+        .then(function(content) {
+            page.content = content;
+        });
+    })
+
+    // Generate template
+    .then(function() {
+        return that.template.renderPage(page);
+    })
+
+    // Prepare and Parse markup
+    .then(function(content) {
+        page.content = content;
+
+        that.log.debug.ln("use file parser", filetype.name, "for", filename);
+        return filetype.page(page.content);
+    })
+
+    // Post process sections
+    .then(function(_page) {
+        return _.reduce(_page.sections, function(prev, section) {
+            return prev.then(function(_sections) {
+                return that.template.postProcess(section.content || "")
+                .then(function(content) {
+                    section.content = content;
+                    return _sections.concat([section]);
+                })
+            });
+        }, Q([]));
+    })
+
+    // Prepare html
+    .then(function(_sections) {
+        return pageUtil.normalize(_sections, {
+            book: that,
+            convertImages: options.convertImages,
+            input: filename,
+            navigation: that.navigation,
+            base: path.dirname(filename) || './',
+            output: path.dirname(filename) || './',
+            glossary: that.glossary
+        });
+    })
+
+    // Interpolate output
+    .then(function(_sections) {
+        page.sections = _sections;
+        return interpolate(options.interpolateContent);
+    })
+
+    .then(function() {
+        that.indexPage(page);
+        return page;
+    });
+};
+
+// Find file that can be parsed with a specific filename
+Book.prototype.findFile = function(filename) {
+    var that = this;
+
+    return _.reduce(parsers.extensions, function(prev, ext) {
+        return prev.then(function(output) {
+            // Stop if already find a parser
+            if (output) return output;
+
+            var filepath = filename+ext;
+
+            return that.fileExists(filepath)
+            .then(function(exists) {
+                if (!exists) return null;
+                return {
+                    parser: parsers.get(ext),
+                    path: filepath
+                };
+            })
+        });
+    }, Q(null));
+};
+
+// Check if a file exists in the book
+Book.prototype.fileExists = function(filename) {
+    return fs.exists(
+        path.join(this.root, filename)
+    );
+};
+
+// Read a file
+Book.prototype.readFile = function(filename) {
+    return fs.readFile(
+        path.join(this.root, filename),
+        { encoding: "utf8" }
+    );
+};
+
+// Return stat for a file
+Book.prototype.statFile = function(filename) {
+    return fs.stat(path.join(this.root, filename));
+};
+
+// List all files in the book
+Book.prototype.listAllFiles = function() {
+    var that = this;
+
+    return fs.list(this.root, {
+        ignoreFiles: ['.ignore', '.gitignore', '.bookignore'],
+        ignoreRules: [
+            // Skip Git stuff
+            '.git/',
+            '.gitignore',
+
+            // Skip OS X meta data
+            '.DS_Store',
+
+            // Skip stuff installed by plugins
+            'node_modules',
+
+            // Skip book outputs
+            '_book',
+            '*.pdf',
+            '*.epub',
+            '*.mobi',
+
+            // Skip config files
+            '.ignore',
+            '.bookignore',
+            'book.json',
+        ]
+    })
+    .then(function(_files) {
+        that.files = _files;
+    });
+};
+
+// Return true if the book is a multilingual book
+Book.prototype.isMultilingual = function(filename) {
+    return this.books.length > 0;
+};
+
+// Return root of the parent
+Book.prototype.parentRoot = function() {
+    if (this.parent) return this.parent.parentRoot();
+    return this.root;
+};
+
+// Return true if it's a sub-book
+Book.prototype.isSubBook = function() {
+    return !!this.parent;
+};
+
+// Test if the file is the entry point
+Book.prototype.isEntryPoint = function(fp) {
+    return fp == this.readmeFile;
+};
+
+// Resolve a path in book
+Book.prototype.resolve = function(p) {
+    return path.resolve(this.root, p);
+};
+
+// Normalize a link to .html and convert README -> index
+Book.prototype.contentLink = function(link) {
+    if (
+        path.basename(link, path.extname(link)) == "README"
+        || link == this.readmeFile
+    ) {
+        link = path.join(path.dirname(link), "index"+path.extname(link));
+    }
+
+    link = links.changeExtension(link, ".html");
+    return link;
+}
+
+// Index a page into the search index
+Book.prototype.indexPage = function(page) {
+    var nav = this.navigation[page.path];
+    if (!nav) return;
+
+    this.log.debug.ln("index page", page.path);
+    this.searchIndex.add({
+        url: this.contentLink(page.path),
+        title: nav.title,
+        body: pageUtil.extractText(page.sections),
+    });
+};
+
+// Default structure paths to an extension
+Book.prototype._defaultsStructure = function(filename) {
+    var that = this;
+    var extension = path.extname(filename);
+
+    that.readmeFile = that.readmeFile || that.config.getStructure("readme")+extension;
+    that.summaryFile = that.summaryFile || that.config.getStructure("summary")+extension;
+    that.glossaryFile = that.glossaryFile || that.config.getStructure("glossary")+extension;
+    that.langsFile = that.langsFile || that.config.getStructure("langs")+extension;
+}
+
+// Change output path
+Book.prototype.setOutput = function(p) {
+    var that = this;
+    this.options.output = path.resolve(p);
+
+    _.each(this.books, function(book) {
+        book.setOutput(path.join(that.options.output, book.options.language));
+    });
+};
+
+
+// Init and return a book
+Book.init = function(root) {
+    var book = new Book(root);
+    var extensionToUse = ".md";
+
+    var chaptersPaths = function(chapters) {
+        return _.reduce(chapters || [], function(accu, chapter) {
+            if (!chapter.path) return accu;
+            return accu.concat(
+                _.filter([
+                    {
+                        title: chapter.title,
+                        path: chapter.path
+                    }
+                ].concat(chaptersPaths(chapter.articles)))
+            );
+        }, []);
+    };
+
+    book.log.info.ln("init book at", root);
+    return fs.mkdirp(root)
+    .then(function() {
+        book.log.info.ln("detect structure from SUMMARY (if it exists)");
+        return book.parseSummary();
+    })
+    .fail(function() {
+        return Q();
+    })
+    .then(function() {
+        var summary = book.summaryFile || "SUMMARY.md";
+        var chapters = book.summary.chapters || [];
+        extensionToUse = path.extname(summary);
+
+        if (chapters.length == 0) {
+            chapters = [
+                {
+                    title: "Summary",
+                    path: "SUMMARY"+extensionToUse
+                },
+                {
+                    title: "Introduction",
+                    path: "README"+extensionToUse
+                }
+            ];
+        }
+
+        return Q(chaptersPaths(chapters));
+    })
+    .then(function(chapters) {
+        // Create files that don't exist
+        return Q.all(_.map(chapters, function(chapter) {
+            var absolutePath = path.resolve(book.root, chapter.path);
+
+            return fs.exists(absolutePath)
+            .then(function(exists) {
+                book.log.info.ln("create", chapter.path);
+                if(exists) return;
+
+                return fs.mkdirp(path.dirname(absolutePath))
+                .then(function() {
+                    return fs.writeFile(absolutePath, '# '+chapter.title+'\n');
+                });
+            });
+        }));
+    })
+    .then(function() {
+        book.log.info.ln("initialization is finished");
+    });
+};
+
+module.exports= Book;

lib/configuration.js

@@ -0,0 +1,243 @@
+var _ = require("lodash");
+var Q = require("q");
+var path = require("path");
+var semver = require('semver');
+
+var pkg = require('../package.json');
+var fs = require("./utils/fs");
+var i18n = require("./utils/i18n");
+
+// Default plugins added to each books
+var defaultsPlugins = [];
+
+// Normalize a list of plugins to use
+function normalizePluginsList(plugins) {
+    // Normalize list to an array
+    plugins = _.isString(plugins) ? plugins.split(",") : (plugins || []);
+
+    // Remove empty parts
+    plugins = _.compact(plugins);
+
+    // Divide as {name, version} to handle format like "myplugin@1.0.0"
+    plugins = _.map(plugins, function(plugin) {
+        if (plugin.name) return plugin;
+
+        var parts = plugin.split("@");
+        return {
+            'name': parts[0],
+            'version': parts[1] // optional
+        }
+    });
+
+    // List plugins to remove
+    var toremove = _.chain(plugins)
+    .filter(function(plugin) {
+        return plugin.name.length > 0 && plugin.name[0] == "-";
+    })
+    .map(function(plugin) {
+        return plugin.name.slice(1);
+    })
+    .value();
+
+    // Merge with defaults
+    plugins = _.chain(plugins)
+    .concat(_.map(defaultsPlugins, function(plugin) {
+        return { 'name': plugin }
+    }))
+    .uniq()
+    .value();
+
+    // Build final list
+    plugins = _.filter(plugins, function(plugin) {
+        return !_.contains(toremove, plugin.name) && !(plugin.name.length > 0 && plugin.name[0] == "-");
+    });
+
+    return plugins;
+}
+
+// Normalize a list of plugin name to use
+function normalizePluginsNames(plugins) {
+    return _.pluck(normalizePluginsList(plugins), "name");
+};
+
+
+var Configuration = function(book, options) {
+    var that = this;
+
+    this.book = book;
+
+    this.options = _.cloneDeep(Configuration.DEFAULT);
+    this.options = _.merge(this.options, options || {});
+
+    // options.input == book.root
+    Object.defineProperty(this.options, "input", {
+        get: function () {
+            return that.book.root;
+        }
+    });
+};
+
+// Read and parse the configuration
+Configuration.prototype.load = function() {
+    var that = this;
+
+    return Q()
+    .then(function() {
+        try {
+            var _config = require(path.resolve(that.book.root, that.options.configFile));
+            that.options = _.merge(
+                that.options,
+                _.omit(_config, 'configFile', 'defaultsPlugins', 'generator', 'extension')
+            );
+        }
+        catch(err) {
+            if (err instanceof SyntaxError) return Q.reject(err);
+            return Q();
+        }
+    })
+    .then(function() {
+        if (!semver.satisfies(pkg.version, that.options.gitbook)) {
+            throw "GitBook version doesn't satisfy version required by the book: "+that.options.gitbook;
+        }
+        if (that.options.gitbook == "*") {
+            that.book.log.warn.ln("you should specify a gitbook version to use in your book.json, for example: "+(_.first(pkg.version.split("."))+".x.x"));
+        }
+
+        that.options.output = path.resolve(that.options.output || path.join(that.book.root, "_book"));
+        that.options.plugins = normalizePluginsList(that.options.plugins);
+        that.options.defaultsPlugins = normalizePluginsList(that.options.defaultsPlugins || "");
+        that.options.plugins = _.union(that.options.plugins, that.options.defaultsPlugins);
+        that.options.plugins = _.uniq(that.options.plugins, "name");
+
+        // Default value for text direction (from language)
+        if (!that.options.direction) {
+            var lang = i18n.getByLanguage(that.options.language);
+            if (lang) that.options.direction = lang.direction;
+        }
+
+        that.options.gitbook = pkg.version;
+    });
+};
+
+// Extend the configuration
+Configuration.prototype.extend = function(options) {
+    _.extend(this.options, options);
+};
+
+// Get structure file
+Configuration.prototype.getStructure = function(name) {
+    return this.options.structure[name].split(".").slice(0, -1).join(".");
+};
+
+// Return normalized language
+Configuration.prototype.normalizeLanguage = function() {
+    return i18n.normalizeLanguage(this.options.language);
+};
+
+
+// Default configuration
+Configuration.DEFAULT = {
+    // Options that can't be extend
+    "configFile": "book",
+    "generator": "website",
+    "extension": null,
+
+    // Book metadats (somes are extracted from the README by default)
+    "title": null,
+    "description": null,
+    "isbn": null,
+    "language": "en",
+    "direction": null,
+    "author": null,
+
+    // version of gitbook to use
+    "gitbook": "*",
+
+    // Structure
+    "structure": {
+        "langs": "LANGS.md",
+        "readme": "README.md",
+        "glossary": "GLOSSARY.md",
+        "summary": "SUMMARY.md"
+    },
+
+    // CSS Styles
+    "styles": {
+        "website": "styles/website.css",
+        "ebook": "styles/ebook.css",
+        "pdf": "styles/pdf.css",
+        "mobi": "styles/mobi.css",
+        "epub": "styles/epub.css"
+    },
+
+    // Plugins list, can contain "-name" for removing default plugins
+    "plugins": [],
+
+    // Global configuration for plugins
+    "pluginsConfig": {
+        "fontSettings": {
+            "theme": null, //"sepia", "night" or "white",
+            "family": "sans",// "serif" or "sans",
+            "size": 2 // 1 - 4
+        }
+    },
+
+    // Variables for templating
+    "variables": {},
+
+    // Set another theme with your own layout
+    // It's recommended to use plugins or add more options for default theme, though
+    // See https://github.com/GitbookIO/gitbook/issues/209
+    "theme": path.resolve(__dirname, '../theme'),
+
+    // Links in template (null: default, false: remove, string: new value)
+    "links": {
+        // Custom links at top of sidebar
+        "sidebar": {
+            //"Custom link name": "https://customlink.com"
+        },
+
+        // Sharing links
+        "sharing": {
+            "google": null,
+            "facebook": null,
+            "twitter": null,
+            "weibo": null,
+            "all": null
+        }
+    },
+
+
+    // Options for PDF generation
+    "pdf": {
+        // Add toc at the end of the file
+        "toc": true,
+
+        // Add page numbers to the bottom of every page
+        "pageNumbers": false,
+
+        // Font size for the file content
+        "fontSize": 12,
+
+        // Paper size for the pdf
+        // Choices are [u’a0’, u’a1’, u’a2’, u’a3’, u’a4’, u’a5’, u’a6’, u’b0’, u’b1’, u’b2’, u’b3’, u’b4’, u’b5’, u’b6’, u’legal’, u’letter’]
+        "paperSize": "a4",
+
+        // Margin (in pts)
+        // Note: 72 pts equals 1 inch
+        "margin": {
+            "right": 62,
+            "left": 62,
+            "top": 56,
+            "bottom": 56
+        },
+
+        //Header HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
+        "headerTemplate": "",
+
+        //Footer HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
+        "footerTemplate": ""
+    }
+};
+
+module.exports= Configuration;

lib/generate/config.js

@@ -1,129 +0,0 @@
-var Q = require('q');
-var _ = require('lodash');
-var path = require('path');
-
-// Default configuration for gitbook
-var CONFIG = {
-    // Folders to use for output
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "output": null,
-
-    // Generator to use for building
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "generator": "site",
-
-    // Configuration file to use
-    "configFile": "book",
-
-    // Book metadats (somes are extracted from the README by default)
-    "title": null,
-    "description": null,
-    "isbn": null,
-
-    // For ebook format, the extension to use for generation (default is detected from output extension)
-    // "epub", "pdf", "mobi"
-    // Caution: it overrides the value from the command line
-    // It's not advised this option in the book.json
-    "extension": null,
-
-    // Plugins list, can contain "-name" for removing default plugins
-    "plugins": [],
-
-    // Global configuration for plugins
-    "pluginsConfig": {
-        "fontSettings": {
-            "theme": null, //"sepia", "night" or "white",
-            "family": "sans",// "serif" or "sans",
-            "size": 2 // 1 - 4
-        }
-    },
-
-    // Variables for templating
-    "variables": {},
-
-    // Set another theme with your own layout
-    // It's recommended to use plugins or add more options for default theme, though
-    // See https://github.com/GitbookIO/gitbook/issues/209
-    "theme": path.resolve(__dirname, '../../theme'),
-
-    // Links in template (null: default, false: remove, string: new value)
-    "links": {
-        // Custom links at top of sidebar
-        "sidebar": {
-            //"Custom link name": "https://customlink.com"
-        },
-
-        // Sharing links
-        "sharing": {
-            "google": null,
-            "facebook": null,
-            "twitter": null,
-            "weibo": null,
-            "all": null
-        }
-    },
-
-
-    // Options for PDF generation
-    "pdf": {
-        // Add toc at the end of the file
-        "toc": true,
-
-        // Add page numbers to the bottom of every page
-        "pageNumbers": false,
-
-        // Font size for the file content
-        "fontSize": 12,
-
-        // Paper size for the pdf
-        // Choices are [u’a0’, u’a1’, u’a2’, u’a3’, u’a4’, u’a5’, u’a6’, u’b0’, u’b1’, u’b2’, u’b3’, u’b4’, u’b5’, u’b6’, u’legal’, u’letter’]
-        "paperSize": "a4",
-
-        // Margin (in pts)
-        // Note: 72 pts equals 1 inch
-        "margin": {
-            "right": 62,
-            "left": 62,
-            "top": 36,
-            "bottom": 36
-        },
-
-        //Header HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
-        "headerTemplate": "",
-
-        //Footer HTML template. Available variables: _PAGENUM_, _TITLE_, _AUTHOR_ and _SECTION_.
-        "footerTemplate": ""
-    }
-};
-
-// Return complete configuration
-var defaultsConfig = function(options) {
-    return _.merge(options || {}, CONFIG, _.defaults);
-};
-
-// Read configuration from book.json
-var readConfig = function(options) {
-    options = defaultsConfig(options);
-
-    return Q()
-    .then(function() {
-        try {
-            var _config = require(path.resolve(options.input, options.configFile));
-            options = _.merge(options, _.omit(_config, 'input', 'configFile', 'defaultsPlugins', 'generator'));
-        }
-        catch(err) {
-            // No config file: not a big deal
-            return Q();
-        }
-    })
-    .thenResolve(options);
-};
-
-module.exports = {
-    CONFIG: CONFIG,
-    defaults: defaultsConfig,
-    read: readConfig
-}
-

lib/generate/fs.js

@@ -1,91 +0,0 @@
-var Q = require("q");
-var fs = require('graceful-fs');
-var fsExtra = require("fs-extra");
-var Ignore = require("fstream-ignore");
-
-var getFiles = function(path) {
-    var d = Q.defer();
-
-    // Our list of files
-    var files = [];
-
-    var ig = Ignore({
-        path: path,
-        ignoreFiles: ['.ignore', '.gitignore', '.bookignore']
-    });
-
-    // Add extra rules to ignore common folders
-    ig.addIgnoreRules([
-        // Skip Git stuff
-        '.git/',
-        '.gitignore',
-
-        // Skip OS X meta data
-        '.DS_Store',
-
-        // Skip stuff installed by plugins
-        'node_modules',
-
-        // Skip book outputs
-        '*.pdf',
-        '*.epub',
-        '*.mobi',
-
-        // Skip config files
-        '.ignore',
-        '.bookignore',
-        'book.json',
-    ], '__custom_stuff');
-
-    // Push each file to our list
-    ig.on('child', function (c) {
-        files.push(
-            c.path.substr(c.root.path.length + 1) + (c.props.Directory === true ? '/' : '')
-        );
-    });
-
-    ig.on('end', function() {
-        // Normalize paths on Windows
-        if(process.platform === 'win32') {
-            return d.resolve(files.map(function(file) {
-                return file.replace(/\\/g, '/');
-            }));
-        }
-
-        // Simply return paths otherwise
-        return d.resolve(files);
-    });
-
-    ig.on('error', d.reject);
-
-    return d.promise;
-};
-
-module.exports = {
-    list: getFiles,
-    readFile: Q.denodeify(fs.readFile),
-    //writeFile: Q.denodeify(fs.writeFile),
-    writeFile: function(filename, data, options) {
-        var d = Q.defer();
-
-        try {
-            fs.writeFileSync(filename, data, options)
-        } catch(err) {
-            d.reject(err);
-        }
-        d.resolve();
-
-
-        return d.promise;
-    },
-    mkdirp: Q.denodeify(fsExtra.mkdirp),
-    copy: Q.denodeify(fsExtra.copy),
-    remove: Q.denodeify(fsExtra.remove),
-    symlink: Q.denodeify(fsExtra.symlink),
-    exists: function(path) {
-        var d = Q.defer();
-        fs.exists(path, d.resolve);
-        return d.promise;
-    },
-    readFileSync: fs.readFileSync.bind(fs)
-};

lib/generate/generator.js

@@ -1,87 +0,0 @@
-var _ = require("lodash");
-var path = require("path");
-var Q = require("q");
-var fs = require("./fs");
-
-var Plugin = require("./plugin");
-
-var BaseGenerator = function(options) {
-    this.options = options;
-
-    // Base for assets in plugins
-    this.pluginAssetsBase = "book";
-
-    this.options.plugins = Plugin.normalizeNames(this.options.plugins);
-    this.options.plugins = _.union(this.options.plugins, this.options.defaultsPlugins);
-    this.plugins = [];
-};
-
-BaseGenerator.prototype.callHook = function(name, data) {
-    return this.plugins.hook(name, data);
-};
-
-// Sets up generator
-BaseGenerator.prototype.load = function() {
-    return this.loadPlugins();
-};
-
-BaseGenerator.prototype.loadPlugins = function() {
-    var that = this;
-
-    return Plugin.fromList(this.options.plugins, this.options.input, this, {
-        assetsBase: this.pluginAssetsBase
-    })
-    .then(function(_plugins) {
-        that.plugins = _plugins;
-
-        return that.callHook("init");
-    });
-};
-
-BaseGenerator.prototype.convertFile = function(content, input) {
-    return Q.reject(new Error("Could not convert "+input));
-};
-
-BaseGenerator.prototype.transferFile = function(input) {
-    return fs.copy(
-        path.join(this.options.input, input),
-        path.join(this.options.output, input)
-    );
-};
-
-BaseGenerator.prototype.transferFolder = function(input) {
-    return fs.mkdirp(
-        path.join(this.options.output, input)
-    );
-};
-
-BaseGenerator.prototype.copyCover = function() {
-    var that = this;
-
-    return Q.all([
-        fs.copy(path.join(this.options.input, "cover.jpg"), path.join(this.options.output, "cover.jpg")),
-        fs.copy(path.join(this.options.input, "cover_small.jpg"), path.join(this.options.output, "cover_small.jpg"))
-    ])
-    .fail(function() {
-        // If orignally from multi-lang, try copy from originalInput
-        if (!that.options.originalInput) return;
-
-        return Q.all([
-            fs.copy(path.join(that.options.originalInput, "cover.jpg"), path.join(that.options.output, "cover.jpg")),
-            fs.copy(path.join(that.options.originalInput, "cover_small.jpg"), path.join(that.options.output, "cover_small.jpg"))
-        ]);
-    })
-    .fail(function(err) {
-        return Q();
-    });
-};
-
-BaseGenerator.prototype.langsIndex = function(langs) {
-    return Q.reject(new Error("Langs index is not supported in this generator"));
-};
-
-BaseGenerator.prototype.finish = function() {
-    return Q.reject(new Error("Could not finish generation"));
-};
-
-module.exports = BaseGenerator;

lib/generate/index.js

@@ -1,374 +0,0 @@
-var Q = require("q");
-var _ = require("lodash");
-var path = require("path");
-var tmp = require('tmp');
-
-var swig = require('./template');
-var fs = require("./fs");
-var parse = require("../parse");
-var Plugin = require("./plugin");
-var defaultConfig = require("./config");
-
-var generators = {
-    "site": require("./site"),
-    "page": require("./page"),
-    "ebook": require("./ebook"),
-    "json": require("./json")
-};
-
-var defaultDescription = "Book generated using GitBook";
-
-
-var containsFiles = function(dir, files) {
-    return Q.all(_.map(files, function(file) {
-        return fs.exists(path.join(dir, file));
-    }))
-    .then(_.all);
-};
-
-// Test if generator exists
-var checkGenerator = function(options) {
-    if (!generators[options.generator]) {
-        return Q.reject(new Error("Invalid generator (availables are: "+_.keys(generators).join(", ")+")"));
-    }
-    return Q();
-};
-
-// Create the generator and load plugins
-var loadGenerator = function(options) {
-    return checkGenerator(options)
-    .then(function() {
-        var generator = new generators[options.generator](options);
-
-        return generator.load()
-        .then(_.constant(generator));
-    });
-};
-
-
-
-var generate = function(options) {
-    // Read config file
-    return defaultConfig.read(options)
-    .then(function(_options) {
-        options = _options;
-
-        // Validate options
-        if (!options.input) {
-            return Q.reject(new Error("Need option input (book input directory)"));
-        }
-
-        // Check files to get folder type (book, multilanguage book or neither)
-        return checkGenerator(options);
-    })
-
-    // Read readme
-    .then(function() {
-        return fs.readFile(path.join(options.input, "README.md"), "utf-8")
-        .then(function(_readme) {
-            _readme = parse.readme(_readme);
-
-            options.title = options.title || _readme.title;
-            options.description = options.description || _readme.description || defaultDescription;
-        });
-    })
-
-    // Detect multi-languages book
-    .then(function() {
-        return containsFiles(options.input, ['LANGS.md'])
-    })
-
-    .then(function(isMultiLang) {
-        // Multi language book
-        if(isMultiLang) {
-            return generateMultiLang(options);
-        }
-
-        // Book
-        return generateBook(options);
-    });
-};
-
-/*
- *  Generate a multilanguage book by generating a book for each folder.
- */
-var generateMultiLang = function(options) {
-    var langsSummary;
-    options.output = options.output || path.join(options.input, "_book");
-
-    return checkGenerator(options)
-
-    // Multi-languages book
-    .then(function() {
-        return fs.readFile(path.join(options.input, "LANGS.md"), "utf-8")
-    })
-
-    // Clean output folder
-    .then(function(_langsSummary) {
-        langsSummary = _langsSummary;
-        return fs.remove(options.output);
-    })
-    .then(function() {
-        return fs.mkdirp(options.output);
-    })
-
-    // Generate sub-books
-    .then(function() {
-        options.langsSummary = parse.langs(langsSummary);
-
-        // Generated a book for each valid entry
-        return _.reduce(options.langsSummary.list, function(prev, entry) {
-            return prev.then(function() {
-                return generate(_.extend({}, options, {
-                    input: path.join(options.input, entry.path),
-                    output: path.join(options.output, entry.path),
-                    originalInput: options.input,
-                    originalOutput: options.output
-                }));
-            })
-        }, Q());
-    })
-
-    .then(function() {
-        return loadGenerator(options);
-    })
-
-    // Generate languages index
-    .then(function(generator) {
-        return generator.langsIndex(options.langsSummary);
-    })
-
-    // Copy cover file
-    .then(function() {
-        return Q.all([
-            fs.copy(path.join(options.input, "cover.jpg"), path.join(options.output, "cover.jpg")),
-            fs.copy(path.join(options.input, "cover_small.jpg"), path.join(options.output, "cover_small.jpg"))
-        ])
-        .fail(function() {
-            return Q();
-        })
-    })
-
-    // Return options to caller
-    .then(_.constant(options));
-};
-
-/*
- *  Use a specific generator to convert a gitbook to a site/pdf/ebook/
- *  output is always a folder
- */
-var generateBook = function(options) {
-    var files;
-
-    options.output = options.output || path.join(options.input, "_book");
-
-    // Check if it's a book
-    return containsFiles(options.input, ['SUMMARY.md', 'README.md'])
-
-    // Fail if not a book
-    .then(function(isBook) {
-        if(!isBook) {
-            return Q.reject(new Error("Invalid gitbook repository, need SUMMARY.md and README.md"));
-        }
-    })
-
-    // Clean output folder
-    .then(function() {
-        return fs.remove(options.output);
-    })
-
-    .then(function() {
-        return fs.mkdirp(options.output);
-    })
-
-    // List all files in the repository
-    .then(function() {
-        return fs.list(options.input)
-        .then(function(_files) {
-            files = _files;
-        });
-    })
-
-    .then(function() {
-        return loadGenerator(options);
-    })
-
-    // Convert files
-    .then(function(generator) {
-        // Generate the book
-        return Q()
-
-        // Get summary
-        .then(function() {
-            var summary = {
-                path: path.join(options.input, "SUMMARY.md")
-            };
-
-            var _callHook = function(name) {
-                return generator.callHook(name, summary)
-                .then(function(_summary) {
-                    summary = _summary;
-                    return summary;
-                });
-            };
-
-            return fs.readFile(summary.path, "utf-8")
-                .then(function(_content) {
-                    summary.content = _content;
-                    return _callHook("summary:before");
-                })
-                .then(function() {
-                    summary.content = parse.summary(summary.content);
-                    return _callHook("summary:after");
-                })
-                .then(function() {
-                    options.summary = summary.content;
-                    options.navigation = parse.navigation(options.summary);
-                });
-        })
-
-        // Read glossary
-        .then(function() {
-            var glossary = {};
-
-            var _callHook = function(name) {
-                return generator.callHook(name, glossary)
-                .then(function(_glossary) {
-                    glossary = _glossary;
-                    return glossary;
-                });
-            };
-
-            return fs.readFile(path.join(options.input, "GLOSSARY.md"), "utf-8")
-                .fail(function() {
-                    return "";
-                })
-                .then(function(_content) {
-                    glossary.content = _content;
-                    return _callHook("glossary:before");
-                })
-                .then(function() {
-                    glossary.content = parse.glossary(glossary.content);
-                    return _callHook("glossary:after");
-                })
-                .then(function() {
-                    options.glossary = glossary.content;
-                });
-        })
-
-        // Skip processing some files
-        .then(function() {
-            files = _.filter(files, function (file) {
-                return !(
-                    file === 'SUMMARY.md' ||
-                    file === 'GLOSSARY.md'
-                );
-            });
-        })
-
-        // Copy file and replace markdown file
-        .then(function() {
-            return Q.all(
-                _.chain(files)
-                .map(function(file) {
-                    if (!file) return;
-
-                    if (file[file.length -1] == "/") {
-                        return Q(generator.transferFolder(file));
-                    } else if (path.extname(file) == ".md" && options.navigation[file] != null) {
-                        return fs.readFile(path.join(options.input, file), "utf-8")
-                        .then(function(content) {
-                            return Q(generator.convertFile(content, file));
-                        });
-                    } else {
-                        return Q(generator.transferFile(file));
-                    }
-                })
-                .value()
-            );
-        })
-
-        // Finish generation
-        .then(function() {
-            return generator.callHook("finish:before");
-        })
-        .then(function() {
-            return generator.finish();
-        })
-        .then(function() {
-            return generator.callHook("finish");
-        });
-    })
-
-    // Return all options
-    .then(function() {
-        return options;
-    });
-};
-
-/*
- *  Extract files from generate output in a temporary folder
- */
-var generateFile = function(options) {
-    options = _.defaults(options || {}, {
-        input: null,
-        output: null,
-        extension: null
-    });
-
-    return Q.nfcall(tmp.dir)
-    .then(function(tmpDir) {
-        return generate(
-            _.extend({},
-                options,
-                {
-                    output: tmpDir
-                })
-        )
-        .then(function(_options) {
-            var ext = options.extension;
-            var outputFile = options.output || path.resolve(options.input, "book."+ext);
-
-            var copyFile = function(lang) {
-                var _outputFile = outputFile;
-                var _tmpDir = tmpDir;
-
-                if (lang) {
-                    _outputFile = _outputFile.slice(0, -path.extname(_outputFile).length)+"_"+lang+path.extname(_outputFile);
-                    _tmpDir = path.join(_tmpDir, lang);
-                }
-
-                return fs.copy(
-                    path.join(_tmpDir, "index."+ext),
-                    _outputFile
-                );
-            };
-
-            // Multi-langs book
-            return Q()
-            .then(function() {
-                if (_options.langsSummary) {
-                    return Q.all(
-                        _.map(_options.langsSummary.list, function(lang) {
-                            return copyFile(lang.lang);
-                        })
-                    );
-                } else {
-                    return copyFile();
-                }
-            })
-            .then(function() {
-                return fs.remove(tmpDir);
-            });
-        });
-    });
-};
-
-module.exports = {
-    generators: generators,
-    folder: generate,
-    file: generateFile,
-    book: generateBook,
-    Plugin: Plugin,
-    config: defaultConfig
-};

lib/generate/init.js

@@ -1,69 +0,0 @@
-var Q = require('q');
-var _ = require('lodash');
-
-var path = require('path');
-
-var fs = require('./fs');
-var parse = require('../parse');
-
-
-// Extract paths out of a summary
-function paths(summary) {
-    return _.reduce(summary.chapters, function(accu, chapter) {
-        return accu.concat(
-            _.filter([chapter.path].concat(_.pluck(chapter.articles, 'path')))
-        );
-    }, []);
-}
-
-// Get the parent folders out of a group of files
-function folders(files) {
-    return _.chain(files)
-    .map(function(file) {
-        return path.dirname(file);
-    })
-    .uniq()
-    .value();
-}
-
-function initDir(dir) {
-    return fs.readFile(path.join(dir, 'SUMMARY.md'), 'utf8')
-    .then(function(src) {
-        // Parse summary
-        return parse.summary(src);
-    })
-    .then(function(summary) {
-        // Extract paths from summary
-        return paths(summary);
-    })
-    .then(function(paths) {
-        // Convert to absolute paths
-        return _.map(paths, function(file) {
-            return path.resolve(file);
-        });
-    })
-    .then(function(files) {
-        // Create folders
-        return Q.all(_.map(folders(files), function(folder) {
-            return fs.mkdirp(folder);
-        }))
-        .then(_.constant(files));
-    })
-    .then(function(files) {
-        // Create files that don't exist
-        return Q.all(_.map(files, function(file) {
-            return fs.exists(file)
-            .then(function(exists) {
-                if(exists) return;
-                return fs.writeFile(file, '');
-            });
-        }));
-    })
-    .fail(function(err) {
-        console.error(err.stack);
-    });
-}
-
-
-// Exports
-module.exports = initDir;

lib/generate/json/index.js

@@ -1,77 +0,0 @@
-var util = require("util");
-var path = require("path");
-var Q = require("q");
-var _ = require("lodash");
-
-var fs = require("../fs");
-var parse = require("../../parse");
-var BaseGenerator = require("../generator");
-
-
-var Generator = function() {
-    BaseGenerator.apply(this, arguments);
-};
-util.inherits(Generator, BaseGenerator);
-
-Generator.prototype.transferFile = function(input) {
-    // ignore
-};
-
-Generator.prototype.convertFile = function(content, input) {
-    var that = this;
-    var json = {
-        progress: parse.progress(this.options.navigation, input)
-    };
-
-    return Q()
-    .then(function() {
-        return parse.page(content, {
-            dir: path.dirname(input) || '/'
-        });
-    })
-    .then(function(parsed) {
-        json.lexed = parsed.lexed;
-        json.sections = parsed.sections;
-    })
-    .then(function() {
-        return fs.writeFile(
-            path.join(that.options.output, input.replace(".md", ".json")),
-            JSON.stringify(json, null, 4)
-        );
-    });
-};
-
-// Generate languages index
-// Contains the first languages readme and langs infos
-Generator.prototype.langsIndex = function(langs) {
-    var that = this;
-
-    if (langs.list.length == 0) return Q.reject("Need at least one language");
-
-    var mainLang = _.first(langs.list).lang;
-    console.log("Main language is", mainLang);
-
-    return Q()
-    .then(function() {
-        return fs.readFile(
-            path.join(that.options.output, mainLang, "README.json")
-        );
-    })
-    .then(function(content) {
-        var json = JSON.parse(content);
-        _.extend(json, {
-            langs: langs.list
-        });
-
-        return fs.writeFile(
-            path.join(that.options.output, "README.json"),
-            JSON.stringify(json, null, 4)
-        );
-    });
-};
-
-Generator.prototype.finish = function() {
-    // ignore
-};
-
-module.exports = Generator;

lib/generate/page/index.js

@@ -1,81 +0,0 @@
-var _ = require("lodash");
-var util = require("util");
-var path = require("path");
-var Q = require("q");
-var swig = require("../template");
-
-var fs = require("../fs");
-var parse = require("../../parse");
-var BaseGenerator = require("../site");
-
-var Generator = function() {
-    BaseGenerator.apply(this, arguments);
-
-    // Base for assets in plugins
-    this.pluginAssetsBase = "ebook";
-
-    // List of pages content
-    this.pages = {};
-};
-util.inherits(Generator, BaseGenerator);
-
-Generator.prototype.loadTemplates = function() {
-    this.template = swig.compileFile(
-        this.plugins.template("ebook:page") || path.resolve(this.options.theme, 'templates/ebook/page.html')
-    );
-    this.summaryTemplate = swig.compileFile(
-        this.plugins.template("ebook:sumary") || path.resolve(this.options.theme, 'templates/ebook/summary.html')
-    );
-    this.glossaryTemplate = swig.compileFile(
-        this.plugins.template("ebook:glossary") || path.resolve(this.options.theme, 'templates/ebook/glossary.html')
-    );
-};
-
-// Generate table of contents
-Generator.prototype.writeToc = function() {
-    var that = this;
-    var basePath = ".";
-
-    return this._writeTemplate(this.summaryTemplate, {
-        toc: parse.progress(this.options.navigation, "README.md").chapters,
-        basePath: basePath,
-        staticBase: path.join(basePath, "gitbook"),
-    }, path.join(this.options.output, "SUMMARY.html"));
-};
-
-Generator.prototype.finish = function() {
-    var that = this;
-    var basePath = ".";
-    var output = path.join(this.options.output, "index.html");
-
-    var progress = parse.progress(this.options.navigation, "README.md");
-    
-    return Q()
-
-    // Write table of contents
-    .then(function() {
-        return that.writeToc();
-    })
-
-    // Write glossary
-    .then(function() {
-        return that.writeGlossary();
-    })
-
-    // Copy cover
-    .then(function() {
-        return that.copyCover();
-    })
-
-    // Copy assets
-    .then(function() {
-        return that.copyAssets();
-    });
-};
-
-// Generate languages index
-Generator.prototype.langsIndex = function(langs) {
-    return Q();
-};
-
-module.exports = Generator;

lib/generate/plugin.js

@@ -1,325 +0,0 @@
-var _ = require("lodash");
-var Q = require("q");
-var semver = require("semver");
-var path = require("path");
-var url = require("url");
-var fs = require("./fs");
-var npmi = require('npmi');
-var resolve = require('resolve');
-
-var pkg = require("../../package.json");
-
-var RESOURCES = ["js", "css"];
-
-var Plugin = function(name, root, generator) {
-    this.name = name;
-    this.root = root;
-    this.packageInfos = {};
-    this.infos = {};
-    this.generator = generator;
-
-    // Bind methods
-    _.bindAll(this);
-
-    _.each([
-        "gitbook-plugin-"+name,
-        "gitbook-theme-"+name,
-        "gitbook-"+name,
-        name,
-    ], function(_name) {
-        if (this.load(_name, __dirname)) return false;
-        if (this.load(_name, path.resolve(root))) return false;
-    }, this);
-};
-
-// Load from a name
-Plugin.prototype.load = function(name, baseDir) {
-    try {
-        var res = resolve.sync(name+"/package.json", { basedir: baseDir });
-
-        this.baseDir = path.dirname(res);
-        this.packageInfos = require(res);
-        this.infos = require(resolve.sync(name, { basedir: baseDir }));
-        this.name = name;
-
-        return true;
-    } catch (e) {
-        return false;
-    }
-};
-
-Plugin.prototype.normalizeResource = function(resource) {
-    // Parse the resource path
-    var parsed = url.parse(resource);
-
-    // This is a remote resource
-    // so we will simply link to using it's URL
-    if (parsed.protocol) {
-        return {
-            "url": resource
-        };
-    }
-
-    // This will be copied over from disk
-    // and shipped with the book's build
-    return { "path": this.name+"/"+resource };
-};
-
-// Return resources
-Plugin.prototype._getResources = function(base) {
-    base = base || "book";
-    var book = this.infos[base];
-
-    // Nothing specified, fallback to default
-    if (!book) {
-        return Q({});
-    }
-
-    // Dynamic function
-    if(typeof book === "function") {
-        // Call giving it the context of our generator
-        return Q().then(book.bind(this.generator));
-    }
-
-    // Plain data object
-    return Q(_.cloneDeep(book));
-};
-
-// Normalize resources and return them
-Plugin.prototype.getResources = function(base) {
-    var that = this;
-
-    return this._getResources(base)
-    .then(function(resources) {
-
-        _.each(RESOURCES, function(resourceType) {
-            resources[resourceType] = (resources[resourceType] || []).map(that.normalizeResource);
-        });
-
-        return resources;
-    });
-};
-
-// Test if it's a valid plugin
-Plugin.prototype.isValid = function() {
-    return (
-        this.packageInfos &&
-        this.packageInfos.name &&
-        this.packageInfos.engines &&
-        this.packageInfos.engines.gitbook &&
-        semver.satisfies(pkg.version, this.packageInfos.engines.gitbook)
-    );
-};
-
-// Resolve file path
-Plugin.prototype.resolveFile = function(filename) {
-    return path.resolve(this.baseDir, filename);
-};
-
-// Resolve file path
-Plugin.prototype.callHook = function(name, data) {
-    // Our generator will be the context to apply
-    var context = this.generator;
-
-    var hookFunc = this.infos.hooks? this.infos.hooks[name] : null;
-    data = data || {};
-
-    if (!hookFunc) return Q(data);
-
-    return Q()
-    .then(function() {
-        return hookFunc.apply(context, [data]);
-    });
-};
-
-// Copy plugin assets fodler
-Plugin.prototype.copyAssets = function(out, options) {
-    var that = this;
-    options = _.defaults(options || {}, {
-        base: "book"
-    });
-
-    return this.getResources(options.base)
-    .get('assets')
-    .then(function(assets) {
-        // Assets are undefined
-        if(!assets) return false;
-
-        return fs.copy(
-            that.resolveFile(assets),
-            out
-        ).then(_.constant(true));
-    }, _.constant(false));
-};
-
-
-// Install a list of plugin
-Plugin.install = function(options) {
-    // Normalize list of plugins
-    var plugins = Plugin.normalizeList(options.plugins);
-
-    // Install plugins one by one
-    return _.reduce(plugins, function(prev, plugin) {
-        return prev.then(function() {
-            var fullname = "gitbook-plugin-"+plugin.name;
-            console.log("Install plugin", plugin.name, "from npm ("+fullname+") with version", (plugin.version || "*"));
-            return Q.nfcall(npmi, {
-                'name': fullname,
-                'version': plugin.version,
-                'path': options.input,
-                'npmLoad': {
-                    'loglevel': 'silent',
-                    'loaded': false,
-                    'prefix': options.input
-                }
-            });
-        });
-    }, Q());
-};
-
-// Normalize a list of plugins to use
-Plugin.normalizeList = function(plugins) {
-    // Normalize list to an array
-    plugins = _.isString(plugins) ? plugins.split(",") : (plugins || []);
-
-    // Divide as {name, version} to handle format like "myplugin@1.0.0"
-    plugins = _.map(plugins, function(plugin) {
-        var parts = plugin.split("@");
-        return {
-            'name': parts[0],
-            'version': parts[1] // optional
-        }
-    });
-
-    // List plugins to remove
-    var toremove = _.chain(plugins)
-    .filter(function(plugin) {
-        return plugin.name.length > 0 && plugin.name[0] == "-";
-    })
-    .map(function(plugin) {
-        return plugin.name.slice(1);
-    })
-    .value();
-
-    // Merge with defaults
-    plugins = _.chain(plugins)
-    .concat(_.map(Plugin.defaults, function(plugin) {
-        return { 'name': plugin }
-    }))
-    .uniq()
-    .value();
-
-    // Build final list
-    plugins = _.filter(plugins, function(plugin) {
-        return !_.contains(toremove, plugin.name) && !(plugin.name.length > 0 && plugin.name[0] == "-");
-    });
-
-    return plugins;
-};
-
-// Normalize a list of plugin name to use
-Plugin.normalizeNames = function(plugins) {
-    return _.pluck(Plugin.normalizeList(plugins), "name");
-};
-
-// Extract data from a list of plugin
-Plugin.fromList = function(names, root, generator, options) {
-    options = _.defaults(options || {}, {
-        assetsBase: "book"
-    });
-
-    var failed = [];
-
-    // Load plugins
-    var plugins = _.map(names, function(name) {
-        var plugin = new Plugin(name, root, generator);
-        if (!plugin.isValid()) failed.push(name);
-        return plugin;
-    });
-
-    if (_.size(failed) > 0) return Q.reject(new Error("Error loading plugins: "+failed.join(",")+". Run 'gitbook install' to install plugins from NPM."));
-
-    // The raw resources extracted from each plugin
-    var pluginResources;
-
-    // Get resources of plugins
-    return Q.all(_.map(plugins, function(plugin) {
-        return plugin.getResources(options.assetsBase);
-    }))
-    // Extract resources out
-    // css, js, etc ...
-    .then(function(resources) {
-        pluginResources = resources;
-        // Group by resource types
-        return _.chain(RESOURCES)
-        .map(function(resourceType) {
-            // Get resources from all the plugins for this current type
-            return [
-                // Key
-                resourceType,
-                // Value
-                _.chain(resources)
-                .pluck(resourceType)
-                .compact()
-                .flatten()
-                .value()
-            ];
-        })
-        .object()
-        .value();
-    })
-    // Extract html snippets
-    .then(function(resources) {
-        // Map of html resources by name added by each plugin
-        resources.html = pluginResources.reduce(function(accu, resource) {
-            var html = (resource && resource.html) || {};
-            _.each(html, function(code, key) {
-                // Turn into function if not one already
-                if (!_.isFunction(code)) code = _.constant(code);
-                // Append
-                accu[key] = (accu[key] || []).concat([code]);
-            });
-
-            return accu;
-        }, {});
-
-        return resources;
-    })
-    // Return big multi-plugin object
-    .then(function(resources) {
-        return {
-            'list': plugins,
-            'resources': resources,
-            'hook': function(name, data) {
-                return _.reduce(plugins, function(prev, plugin) {
-                    return prev.then(function(ret) {
-                        return plugin.callHook(name, ret);
-                    });
-                }, Q(data));
-            },
-            'template': function(name) {
-                var withTpl = _.find(plugins, function(plugin) {
-                    return (
-                        plugin.infos.templates &&
-                        plugin.infos.templates[name]
-                    );
-                });
-
-                if (!withTpl) return null;
-                return withTpl.resolveFile(withTpl.infos.templates[name]);
-            },
-            'html': function(tag, context, options) {
-                return _.map(resources.html[tag] || [], function(code) {
-                    return code.call(context, options);
-                }).join("\n");
-            }
-        };
-    });
-};
-
-// Default plugins added to each books
-Plugin.defaults = [
-    "mathjax"
-];
-
-module.exports = Plugin;

lib/generate/site/glossary_indexer.js

@@ -1,101 +0,0 @@
-var _ = require("lodash");
-var kramed = require('kramed');
-var textRenderer = require('kramed-text-renderer');
-
-var entryId = require('../../parse/glossary').entryId;
-
-
-function Indexer(glossary) {
-    if(!(this instanceof Indexer)) {
-        return new Indexer(glossary);
-    }
-
-    _.bindAll(this);
-
-    this.glossary = glossary || [];
-
-    this.glossaryTerms = _.pluck(this.glossary, "id");
-
-    // Regex for searching for terms through body
-    this.termsRegex = new RegExp(
-        // Match any of the terms
-        "("+
-            this.glossaryTerms.map(regexEscape).join('|') +
-        ")",
-
-        // Flags
-        "gi"
-    );
-
-    // page url => terms
-    this.idx = {
-        /*
-        "a/b.html": ["one word", "second word"]
-        */
-    };
-
-    // term => page urls
-    this.invertedIdx = {
-        /*
-        "word1": ["page1.html", "page2.html"]
-        */
-    };
-
-    // Use text renderer
-    this.renderer = textRenderer();
-}
-
-Indexer.prototype.text = function(nodes) {
-    // Copy section
-    var section = _.toArray(nodes);
-
-    // kramed's Render expects this, we don't use it yet
-    section.links = {};
-
-    var options = _.extend({}, kramed.defaults, {
-        renderer: this.renderer
-    });
-
-    return kramed.parser(section, options);
-};
-
-// Add page to glossary index
-Indexer.prototype.add = function(sections, url) {
-    if(!(this.glossary && this.glossary.length > 0)) {
-        return;
-    }
-
-    var textblob =
-    _.where(sections, { type: 'normal' })
-    .map(this.text)
-    .join('\n');
-
-    var matches = _(textblob.match(this.termsRegex) || [])
-    .map(entryId)
-    .uniq()
-    .value();
-
-    // Add idx for book
-    this.idx[url] = matches;
-
-    // Add to inverted idx
-    matches.forEach(function(match) {
-        if(!this.invertedIdx[match]) {
-            this.invertedIdx[match] = [];
-        }
-        this.invertedIdx[match].push(url);
-    }.bind(this));
-};
-
-// Dump index as a string
-Indexer.prototype.dump = function() {
-    return JSON.stringify(this.idx);
-};
-
-
-function regexEscape(s) {
-    return s.replace(/[-\/\\^$*+?.()|[\]{}]/g, '\\$&');
-}
-
-// Exports
-module.exports = Indexer;

lib/generate/site/index.js

@@ -1,293 +0,0 @@
-var util = require("util");
-var path = require("path");
-var Q = require("q");
-var _ = require("lodash");
-var swig = require("../template");
-
-var fs = require("../fs");
-var parse = require("../../parse");
-var BaseGenerator = require("../generator");
-var links = require("../../utils/links");
-var indexer = require('./search_indexer');
-var glossaryIndexer = require('./glossary_indexer');
-
-
-var Generator = function() {
-    BaseGenerator.apply(this, arguments);
-
-    // Attach methods to instance
-    _.bindAll(this);
-
-    this.revision = Date.now();
-    this.indexer = indexer();
-};
-util.inherits(Generator, BaseGenerator);
-
-// Add template loading to load
-Generator.prototype.load = function() {
-    var that = this;
-
-    return BaseGenerator.prototype.load.apply(this)
-    .then(function() {
-        return that.loadTemplates();
-    });
-};
-
-// Load all templates
-Generator.prototype.loadTemplates = function() {
-    this.template = swig.compileFile(
-        this.plugins.template("site:page") || path.resolve(this.options.theme, 'templates/book/page.html')
-    );
-    this.langsTemplate = swig.compileFile(
-        this.plugins.template("site:langs") || path.resolve(this.options.theme, 'templates/book/langs.html')
-    );
-    this.glossaryTemplate = swig.compileFile(
-        this.plugins.template("site:glossary") || path.resolve(this.options.theme, 'templates/book/glossary.html')
-    );
-};
-
-// Generate a template
-Generator.prototype._writeTemplate = function(tpl, options, output, interpolate) {
-    var that = this;
-
-    interpolate = interpolate || _.identity;
-    return Q()
-    .then(function(sections) {
-        return tpl(_.extend({
-            revision: that.revision,
-
-            title: that.options.title,
-            description: that.options.description,
-
-            glossary: that.options.glossary,
-
-            summary: that.options.summary,
-            allNavigation: that.options.navigation,
-
-            plugins: that.plugins,
-            pluginsConfig: JSON.stringify(that.options.pluginsConfig),
-            htmlSnippet: _.partialRight(that.plugins.html, that, options),
-
-            options: that.options
-        }, options));
-    })
-    .then(interpolate)
-    .then(function(html) {
-        return fs.writeFile(
-            output,
-            html
-        );
-    });
-};
-
-Generator.prototype.indexPage = function(lexed, pagePath) {
-    // Setup glossary indexer if not yet setup
-    if(!this.glossaryIndexer) {
-        this.glossaryIndexer = glossaryIndexer(this.options.glossary);
-    }
-
-    this.indexer.add(lexed, pagePath);
-    this.glossaryIndexer.add(lexed, pagePath);
-    return Q();
-};
-
-// Convert a markdown file into a normalized data set
-Generator.prototype.prepareFile  = function(content, _input) {
-    var that = this;
-
-    var input = path.join(this.options.input, _input);
-
-    var page = {
-        path: _input,
-        rawPath: input,
-        content: content,
-        progress: parse.progress(this.options.navigation, _input)
-    };
-
-    var _callHook = function(name) {
-        return that.callHook(name, page)
-        .then(function(_page) {
-            page = _page;
-            return page;
-        });
-    };
-
-    return Q()
-    .then(function() {
-        // Send content to plugins
-        return _callHook("page:before");
-    })
-    .then(function() {
-        // Lex, parse includes and get
-        // Get HTML generated sections
-        return parse.page(page.content, {
-            // Local files path
-            dir: path.dirname(_input) || '/',
-
-            // Output directory
-            outdir: path.dirname(_input) || '/',
-
-            // Includer for templating
-            includer: parse.includer(that.options.variables, [
-                    path.dirname(_input) || '/',
-                    path.join(that.options.input, '_includes'),
-            ], path.join, fs.readFileSync)
-        });
-    })
-    .then(function(parsed) {
-        page.lexed = parsed.lexed;
-        page.sections = parsed.sections;
-
-        // Use plugin hook
-        return _callHook("page");
-    })
-    .then(function() {
-        return page;
-    });
-};
-
-// Convert a markdown file to html
-Generator.prototype.convertFile = function(content, _input) {
-    var that = this;
-
-    var _output = _input.replace(".md", ".html");
-    if (_output == "README.html") _output = "index.html";
-    var output = path.join(this.options.output, _output);
-    var basePath = path.relative(path.dirname(output), this.options.output) || ".";
-	
-	// Bug fix for issue #493 which would occur when relative-links are 2-level or more deep in win32
-	if (process.platform === 'win32') {
-		basePath = basePath.replace(/\\/g, '/');
-	}
-	
-    return this.prepareFile(content, _input)
-    .then(function(page) {
-        // Index page in search
-        return that.indexPage(page.lexed, _output).thenResolve(page);
-    })
-    .then(function(page) {
-        // Write file
-        return that._writeTemplate(that.template, {
-            progress: page.progress,
-
-            _input: page.path,
-            content: page.sections,
-
-            basePath: basePath,
-            staticBase: links.join(basePath, "gitbook"),
-        }, output, function(html) {
-            page.content = html;
-
-            return that.callHook("page:after", page).get("content")
-        });
-    });
-};
-
-// Generate languages index
-Generator.prototype.langsIndex = function(langs) {
-    var that = this;
-    var basePath = ".";
-
-    return this._writeTemplate(this.langsTemplate, {
-        langs: langs.list,
-
-        basePath: basePath,
-        staticBase: path.join(basePath, "gitbook"),
-    }, path.join(this.options.output, "index.html"))
-    .then(function() {
-        // Copy assets
-        return that.copyAssets();
-    });
-};
-
-// Generate glossary
-Generator.prototype.writeGlossary = function() {
-    var that = this;
-    var basePath = ".";
-
-    // No glossary
-    if (!this.glossaryIndexer) return Q();
-
-    // Transform the glossary to get term, description, files
-    var glossary = _.chain(this.glossaryIndexer.invertedIdx)
-    .map(function(links, id) {
-        var term = _.find(that.options.glossary, { 'id': id });
-
-        return {
-            id: id,
-            name: term.name,
-            description: term.description,
-            files: _.chain(links)
-                .map(function(link) {
-                    var name = link.slice(0, -5);
-
-                    if (name == "index") {
-                        name = "README";
-                    }
-                    return that.options.navigation[name+".md"];
-                })
-                .sortBy("percent")
-                .value()
-        }
-    })
-    .sortBy("name")
-    .value();
-
-    return this._writeTemplate(this.glossaryTemplate, {
-        glossaryIndex: glossary,
-        basePath: basePath,
-        staticBase: path.join(basePath, "gitbook"),
-    }, path.join(this.options.output, "GLOSSARY.html"));
-};
-
-// Copy assets
-Generator.prototype.copyAssets = function() {
-    var that = this;
-
-    // Copy gitbook assets
-    return fs.copy(
-        path.join(that.options.theme, "assets"),
-        path.join(that.options.output, "gitbook")
-    )
-
-    // Copy plugins assets
-    .then(function() {
-        return Q.all(
-            _.map(that.plugins.list, function(plugin) {
-                var pluginAssets = path.join(that.options.output, "gitbook/plugins/", plugin.name);
-                return plugin.copyAssets(pluginAssets, {
-                    base: that.pluginAssetsBase
-                });
-            })
-        );
-    });
-};
-
-// Dump search index to disk
-Generator.prototype.writeSearchIndex = function() {
-    return fs.writeFile(
-        path.join(this.options.output, 'search_index.json'),
-        this.indexer.dump()
-    );
-};
-
-// Dump glossary index to disk
-Generator.prototype.writeGlossaryIndex = function() {
-    if (!this.glossaryIndexer) return Q();
-
-    return fs.writeFile(
-        path.join(this.options.output, 'glossary_index.json'),
-        JSON.stringify(this.options.glossary)
-    );
-};
-
-
-Generator.prototype.finish = function() {
-    return this.copyAssets()
-    .then(this.copyCover)
-    .then(this.writeGlossary)
-    .then(this.writeGlossaryIndex)
-    .then(this.writeSearchIndex);
-};
-
-module.exports = Generator;

lib/generate/site/search_indexer.js

@@ -1,71 +0,0 @@
-var Q = require("q");
-var _ = require("lodash");
-
-var lunr = require('lunr');
-var kramed = require('kramed');
-var textRenderer = require('kramed-text-renderer');
-
-
-function Indexer() {
-    if(!(this instanceof Indexer)) {
-        return new Indexer();
-    }
-
-    _.bindAll(this);
-
-    // Setup lunr index
-    this.idx = lunr(function () {
-        this.ref('url');
-
-        this.field('title', { boost: 10 });
-        this.field('body');
-    });
-
-    this.renderer = textRenderer();
-}
-
-Indexer.prototype.text = function(nodes) {
-    // Copy section
-    var section = _.toArray(nodes);
-
-    // kramed's Render expects this, we don't use it yet
-    section.links = {};
-
-    var options = _.extend({}, kramed.defaults, {
-        renderer: this.renderer
-    });
-
-    return kramed.parser(section, options);
-};
-
-Indexer.prototype.addSection = function(path, section) {
-    var url = [path, section.id].join('#');
-
-    var title = this.text(
-        _.filter(section, {'type': 'heading'})
-    );
-
-    var body = this.text(
-        _.omit(section, {'type': 'heading'})
-    );
-
-    // Add to lunr index
-    this.idx.add({
-        url: url,
-        title: title,
-        body: body,
-    });
-};
-
-Indexer.prototype.add = function(lexedPage, url) {
-    var sections = lexedPage;
-
-    _.map(sections, _.partial(this.addSection, url));
-};
-
-Indexer.prototype.dump = function() {
-    return JSON.stringify(this.idx);
-};
-
-// Exports
-module.exports = Indexer;

lib/generate/template.js

@@ -1,52 +0,0 @@
-var path = require("path");
-var swig = require('swig');
-var hljs = require('highlight.js');
-
-var links = require('../utils/').links;
-var pkg = require('../../package.json');
-
-swig.setDefaults({
-    locals: {
-        gitbook: {
-            version: pkg.version
-        }
-    }
-});
-
-// Swig filter for returning the count of lines in a code section
-swig.setFilter('lines', function(content) {
-    return content.split('\n').length;
-});
-
-// Swig filter for returning a link to the associated html file of a markdown file
-swig.setFilter('mdLink', function(link) {
-    var link = link.replace(".md", ".html");
-    if (link == "README.html") link = "index.html";
-    return link;
-});
-
-// Swig filter: highlight coloration
-swig.setFilter('code', function(code, lang) {
-    try {
-        return hljs.highlight(lang, code).value;
-    } catch(e) {
-        return hljs.highlightAuto(code).value;
-    }
-});
-
-// Convert a level into a deep level
-swig.setFilter('lvl', function(lvl) {
-    return lvl.split(".").length;
-});
-
-// Join path
-swig.setFilter('pathJoin', function(base, _path) {
-    return links.join(base, _path);
-});
-
-// Is a link an absolute link
-swig.setFilter('isExternalLink', function(link) {
-    return links.isExternal(link);
-});
-
-module.exports = swig;

lib/generator.js

@@ -0,0 +1,78 @@
+var _ = require("lodash");
+var path = require("path");
+var Q = require("q");
+var fs = require("./utils/fs");
+
+var Plugin = require("./plugin");
+
+var BaseGenerator = function(book) {
+    this.book = book;
+
+    Object.defineProperty(this, "options", {
+        get: function () {
+            return this.book.options;
+        }
+    });
+
+    _.bindAll(this);
+};
+
+BaseGenerator.prototype.callHook = function(name, data) {
+    return this.book.plugins.hook(name, data);
+};
+
+// Prepare the genertor
+BaseGenerator.prototype.prepare = function() {
+    var that = this;
+
+    return that.callHook("init");
+};
+
+// Write a parsed file to the output
+BaseGenerator.prototype.convertFile = function(input) {
+    return Q.reject(new Error("Could not convert "+input));
+};
+
+// Copy file to the output (non parsable)
+BaseGenerator.prototype.transferFile = function(input) {
+    return fs.copy(
+        path.join(this.book.root, input),
+        path.join(this.options.output, input)
+    );
+};
+
+// Copy a folder to the output
+BaseGenerator.prototype.transferFolder = function(input) {
+    return fs.mkdirp(
+        path.join(this.book.options.output, input)
+    );
+};
+
+// Copy the cover picture
+BaseGenerator.prototype.copyCover = function() {
+    var that = this;
+
+    return Q.all([
+        fs.copy(path.join(that.book.root, "cover.jpg"), path.join(that.options.output, "cover.jpg")),
+        fs.copy(path.join(that.book.root, "cover_small.jpg"), path.join(that.options.output, "cover_small.jpg"))
+    ])
+    .fail(function() {
+        // If orignaly from multi-lang, try copy from parent
+        if (!that.book.isSubBook()) return;
+
+        return Q.all([
+            fs.copy(path.join(that.book.parentRoot(), "cover.jpg"), path.join(that.options.output, "cover.jpg")),
+            fs.copy(path.join(that.book.parentRoot(), "cover_small.jpg"), path.join(that.options.output, "cover_small.jpg"))
+        ]);
+    })
+    .fail(function(err) {
+        return Q();
+    });
+};
+
+// At teh end of the generation
+BaseGenerator.prototype.finish = function() {
+    return Q.reject(new Error("Could not finish generation"));
+};
+
+module.exports = BaseGenerator;

lib/generators/ebook.js

@@ -4,23 +4,55 @@ var Q = require("q");
 var _ = require("lodash");
 var exec = require('child_process').exec;
 
-var fs = require('graceful-fs');
-var parse = require("../../parse");
-var BaseGenerator = require("../page");
-var stringUtils = require("../../utils/string");
+var fs = require("../utils/fs");
+var stringUtils = require("../utils/string");
+var BaseGenerator = require("./website");
 
-var Generator = function() {
+var Generator = function(book, format) {
     BaseGenerator.apply(this, arguments);
+
+    // eBook format
+    this.ebookFormat = format;
+
+    // Resources namespace
+    this.namespace = "ebook";
+
+    // Styles to use
+    this.styles = _.compact(["ebook", this.ebookFormat]);
+
+    // Convert images (svg -> png)
+    this.convertImages = true;
 };
 util.inherits(Generator, BaseGenerator);
 
+Generator.prototype.prepareTemplates = function() {
+    this.templates["page"] = this.book.plugins.template("ebook:page") || path.resolve(this.options.theme, 'templates/ebook/page.html');
+    this.templates["summary"] = this.book.plugins.template("ebook:summary") || path.resolve(this.options.theme, 'templates/ebook/summary.html');
+    this.templates["glossary"] = this.book.plugins.template("ebook:glossary") || path.resolve(this.options.theme, 'templates/ebook/glossary.html');
+
+    return Q();
+};
+
+// Generate table of contents
+Generator.prototype.writeSummary = function() {
+    var that = this;
+
+    that.book.log.info.ln("write SUMMARY.html");
+    return this._writeTemplate(this.templates["summary"], {}, path.join(this.options.output, "SUMMARY.html"));
+};
+
 Generator.prototype.finish = function() {
     var that = this;
 
-    return BaseGenerator.prototype.finish.apply(this)
+    return Q()
+    .then(this.copyAssets)
+    .then(this.copyCover)
+    .then(this.writeGlossary)
+    .then(this.writeSummary)
     .then(function() {
+        if (!that.ebookFormat) return Q();
+
         var d = Q.defer();
-        var format = that.options.extension || path.extname(that.options.output).replace("\.", "") || "pdf";
 
         if (!that.options.cover && fs.existsSync(path.join(that.options.output, "cover.jpg"))) {
             that.options.cover = path.join(that.options.output, "cover.jpg");
@@ -32,6 +64,8 @@ Generator.prototype.finish = function() {
             "--comments": that.options.description,
             "--isbn": that.options.isbn,
             "--authors": that.options.author,
+            "--language": that.options.language,
+            "--book-producer": "GitBook",
             "--publisher": "GitBook",
             "--chapter": "descendant-or-self::*[contains(concat(' ', normalize-space(@class), ' '), ' book-chapter ')]",
             "--chapter-mark": "pagebreak",
@@ -44,7 +78,7 @@ Generator.prototype.finish = function() {
             "--breadth-first": true
         };
 
-        if (format == "pdf") {
+        if (that.ebookFormat == "pdf") {
             var pdfOptions = that.options.pdf;
 
             _.extend(_options, {
@@ -56,20 +90,28 @@ Generator.prototype.finish = function() {
                 "--pdf-mono-font-size": String(pdfOptions.fontSize),
                 "--paper-size": String(pdfOptions.paperSize),
                 "--pdf-page-numbers": Boolean(pdfOptions.pageNumbers),
-                "--pdf-header-template": String(pdfOptions.headerTemplate),
-                "--pdf-footer-template": String(pdfOptions.footerTemplate)
+                "--pdf-header-template": String(pdfOptions.headerTemplate) || "<p class='header'><span>"+that.options.title+"</span></p>",
+                "--pdf-footer-template": String(pdfOptions.footerTemplate) || "<p class='footer'><span>_SECTION_</span> <span style='float:right;'>_PAGENUM_</span></p>"
+            });
+        } else if (that.ebookFormat == "epub") {
+            _.extend(_options, {
+                "--dont-split-on-page-breaks": true
             });
         }
 
         var command = [
             "ebook-convert",
             path.join(that.options.output, "SUMMARY.html"),
-            path.join(that.options.output, "index."+format),
+            path.join(that.options.output, "index."+that.ebookFormat),
             stringUtils.optionsToShellArgs(_options)
         ].join(" ");
 
-        exec(command, function (error, stdout, stderr) {
+        that.book.log.info("start conversion to", that.ebookFormat, "....");
+
+        var child = exec(command, function (error, stdout, stderr) {
             if (error) {
+                that.book.log.info.fail();
+
                 if (error.code == 127) {
                     error.message = "Need to install ebook-convert from Calibre";
                 } else {
@@ -77,9 +119,19 @@ Generator.prototype.finish = function() {
                 }
                 return d.reject(error);
             }
+
+            that.book.log.info.ok();
             d.resolve();
         });
 
+        child.stdout.on('data', function (data) {
+            that.book.log.debug(data);
+        });
+
+        child.stderr.on('data', function (data) {
+            that.book.log.debug(data);
+        });
+
         return d.promise;
     });
 };

lib/generators/index.js

@@ -0,0 +1,11 @@
+var _ = require("lodash");
+var EbookGenerator = require("./ebook");
+
+module.exports = {
+    json: require("./json"),
+    website: require("./website"),
+    ebook: EbookGenerator,
+    pdf: _.partialRight(EbookGenerator, "pdf"),
+    mobi: _.partialRight(EbookGenerator, "mobi"),
+    epub: _.partialRight(EbookGenerator, "epub")
+};

lib/generators/json.js

@@ -0,0 +1,76 @@
+var util = require("util");
+var path = require("path");
+var Q = require("q");
+var _ = require("lodash");
+
+var fs = require("../utils/fs");
+var BaseGenerator = require("../generator");
+var links = require("../utils/links");
+
+var Generator = function() {
+    BaseGenerator.apply(this, arguments);
+};
+util.inherits(Generator, BaseGenerator);
+
+// Ignore some methods
+Generator.prototype.transferFile = function(input) { };
+
+// Convert an input file
+Generator.prototype.convertFile = function(input) {
+    var that = this;
+
+    return that.book.parsePage(input)
+    .then(function(page) {
+        var json = {
+            progress: page.progress,
+            sections: page.sections
+        };
+
+        var output = links.changeExtension(page.path, ".json");
+        output = path.join(that.options.output, output);
+
+        return fs.writeFile(
+            output,
+            JSON.stringify(json, null, 4)
+        );
+    });
+};
+
+// Finish generation
+Generator.prototype.finish = function() {
+    return this.writeReadme();
+};
+
+// Write README.json
+Generator.prototype.writeReadme = function() {
+    var that = this;
+    var mainlang, langs;
+
+    return Q()
+    .then(function() {
+        langs = that.book.langs;
+        mainLang = langs.length > 0? _.first(langs).lang : null;
+
+        readme = links.changeExtension(that.book.readmeFile, ".json");
+
+        // Read readme from main language
+        return fs.readFile(
+            mainLang? path.join(that.options.output, mainLang, readme) : path.join(that.options.output, readme)
+        );
+    })
+    .then(function(content) {
+        // Extend it with infos about the languages
+        var json = JSON.parse(content);
+        _.extend(json, {
+            langs: langs
+        });
+
+        // Write it as README.json
+        return fs.writeFile(
+            path.join(that.options.output, "README.json"),
+            JSON.stringify(json, null, 4)
+        );
+    });
+};
+
+module.exports = Generator;

lib/generators/website.js

@@ -0,0 +1,279 @@
+var util = require("util");
+var path = require("path");
+var Q = require("q");
+var _ = require("lodash");
+
+var nunjucks = require("nunjucks");
+var AutoEscapeExtension = require("nunjucks-autoescape");
+var FilterExtension = require("nunjucks-filter");
+var I18nExtension = require("nunjucks-i18n");
+
+var fs = require("../utils/fs");
+var BaseGenerator = require("../generator");
+var links = require("../utils/links");
+var pageUtil = require("../utils/page");
+var i18n = require("../utils/i18n");
+
+var pkg = require("../../package.json");
+
+var Generator = function() {
+    BaseGenerator.apply(this, arguments);
+
+    // Revision
+    this.revision = new Date();
+
+    // Resources namespace
+    this.namespace = "website";
+
+    // Style to integrates in the output
+    this.styles = ["website"];
+
+    // Convert images (svg -> png)
+    this.convertImages = false;
+
+    // Templates
+    this.templates = {};
+};
+util.inherits(Generator, BaseGenerator);
+
+// Prepare the genertor
+Generator.prototype.prepare = function() {
+    return BaseGenerator.prototype.prepare.apply(this)
+    .then(this.prepareStyles)
+    .then(this.prepareTemplates)
+    .then(this.prepareTemplateEngine);
+};
+
+// Prepare all styles
+Generator.prototype.prepareStyles = function() {
+    var that = this;
+
+    this.styles = _.chain(this.styles)
+        .map(function(style) {
+            var stylePath = that.options.styles[style];
+            if (fs.existsSync(path.resolve(that.book.root, stylePath))) {
+                return stylePath;
+            }
+            return null;
+        })
+        .compact()
+        .value();
+
+    return Q();
+};
+
+// Prepare templates
+Generator.prototype.prepareTemplates = function() {
+    this.templates["page"] = this.book.plugins.template("site:page") || path.resolve(this.options.theme, 'templates/website/page.html');
+    this.templates["langs"] = this.book.plugins.template("site:langs") || path.resolve(this.options.theme, 'templates/website/langs.html');
+    this.templates["glossary"] = this.book.plugins.template("site:glossary") || path.resolve(this.options.theme, 'templates/website/glossary.html');
+
+    return Q();
+};
+
+// Prepare template engine
+Generator.prototype.prepareTemplateEngine = function() {
+    var that = this;
+
+    return Q()
+    .then(function() {
+        var language = that.book.config.normalizeLanguage();
+
+        if (!i18n.getByLanguage(language) && language != "en") {
+            that.book.log.warn.ln("Language '"+language+"' is not available as a layout locales (en, "+i18n.getLanguages().join(", ")+")");
+        }
+
+        var folders = _.chain(that.templates)
+            .values()
+            .map(path.dirname)
+            .uniq()
+            .value();
+
+        that.env = new nunjucks.Environment(
+            new nunjucks.FileSystemLoader(folders),
+            {
+                autoescape: true
+            }
+        );
+
+        // Add filter
+        that.env.addFilter("contentLink", that.book.contentLink.bind(that.book));
+        that.env.addFilter('lvl', function(lvl) {
+            return lvl.split(".").length;
+        });
+
+        // Add extension
+        that.env.addExtension('AutoEscapeExtension', new AutoEscapeExtension(that.env));
+        that.env.addExtension('FilterExtension', new FilterExtension(that.env));
+        that.env.addExtension('I18nExtension', new I18nExtension({
+            env: that.env,
+            translations: i18n.getLocales(),
+            locale: "language"
+        }));
+    });
+};
+
+// Finis generation
+Generator.prototype.finish = function() {
+    return this.copyAssets()
+    .then(this.copyCover)
+    .then(this.writeGlossary)
+    .then(this.writeSearchIndex)
+    .then(this.writeLangsIndex)
+};
+
+// Convert an input file
+Generator.prototype.convertFile = function(input) {
+    var that = this;
+
+    return that.book.parsePage(input, {
+        convertImages: that.convertImages,
+        interpolateTemplate: function(page) {
+            return that.callHook("page:before", page);
+        },
+        interpolateContent: function(page) {
+            return that.callHook("page", page);
+        }
+    })
+    .then(function(page) {
+        var relativeOutput = that.book.contentLink(page.path);
+        var output = path.join(that.options.output, relativeOutput);
+
+        var basePath = path.relative(path.dirname(output), that.options.output) || ".";
+        if (process.platform === 'win32') basePath = basePath.replace(/\\/g, '/');
+
+        that.book.log.debug.ln("write parsed file", page.path, "to", relativeOutput);
+
+        return that._writeTemplate(that.templates["page"], {
+            progress: page.progress,
+
+            _input: page.path,
+            content: page.sections,
+
+            basePath: basePath,
+            staticBase: links.join(basePath, "gitbook")
+        }, output);
+    });
+};
+
+// Write the index for langs
+Generator.prototype.writeLangsIndex = function() {
+    var that = this;
+    if (!this.book.langs.length) return Q();
+    return this._writeTemplate(this.templates["langs"], {
+        langs: this.book.langs
+    }, path.join(this.options.output, "index.html"));
+};
+
+// Write glossary
+Generator.prototype.writeGlossary = function() {
+    var that = this;
+
+    // No glossary
+    if (this.book.glossary.length == 0) return Q();
+
+    return this._writeTemplate(this.templates["glossary"], {}, path.join(this.options.output, "GLOSSARY.html"));
+};
+
+// Write the search index
+Generator.prototype.writeSearchIndex = function() {
+    var that = this;
+
+    return fs.writeFile(
+        path.join(this.options.output, "search_index.json"),
+        JSON.stringify(this.book.searchIndex)
+    );
+};
+
+// Convert a page into a normalized data set
+Generator.prototype.normalizePage  = function(page) {
+    var that = this;
+
+    var _callHook = function(name) {
+        return that.callHook(name, page)
+        .then(function(_page) {
+            page = _page;
+            return page;
+        });
+    };
+
+    return Q()
+    .then(function() {
+        return _callHook("page");
+    })
+    .then(function() {
+        return page;
+    });
+};
+
+// Generate a template
+Generator.prototype._writeTemplate = function(tpl, options, output, interpolate) {
+    var that = this;
+
+    interpolate = interpolate || _.identity;
+    return Q()
+    .then(function(sections) {
+        return that.env.render(
+            tpl,
+            _.extend({
+                gitbook: {
+                    version: pkg.version
+                },
+
+                styles: that.styles,
+
+                revision: that.revision,
+
+                title: that.options.title,
+                description: that.options.description,
+                language: that.book.config.normalizeLanguage(),
+
+                glossary: that.book.glossary,
+
+                summary: that.book.summary,
+                allNavigation: that.book.navigation,
+
+                plugins: {
+                    resources: that.book.plugins.resources(that.namespace)
+                },
+                pluginsConfig: JSON.stringify(that.options.pluginsConfig),
+                htmlSnippet: _.partial(_.partialRight(that.book.plugins.html, that, options), that.namespace),
+
+                options: that.options,
+
+                basePath: ".",
+                staticBase: path.join(".", "gitbook"),
+            }, options)
+        );
+    })
+    .then(interpolate)
+    .then(function(html) {
+        return fs.writeFile(
+            output,
+            html
+        );
+    });
+};
+
+// Copy assets
+Generator.prototype.copyAssets = function() {
+    var that = this;
+
+    // Copy gitbook assets
+    return fs.copy(
+        path.join(that.options.theme, "assets"),
+        path.join(that.options.output, "gitbook")
+    )
+
+    // Copy plugins assets
+    .then(function() {
+        return Q.all(
+            _.map(that.book.plugins.list, function(plugin) {
+                var pluginAssets = path.join(that.options.output, "gitbook/plugins/", plugin.name);
+                return plugin.copyAssets(pluginAssets, that.namespace);
+            })
+        );
+    });
+};
+
+module.exports = Generator;

lib/index.js

@@ -1,5 +1,210 @@
-module.exports = {
-    parse: require('./parse/'),
-    generate: require('./generate/')
+var Q = require("q");
+var _ = require("lodash");
+var path = require("path");
+var tinylr = require('tiny-lr');
+var color = require('bash-color');
+
+var Book = require("./book");
+var Plugin = require("./plugin");
+var Server = require("./utils/server");
+var stringUtils = require("./utils/string");
+var watch = require("./utils/watch");
+var logger = require("./utils/logger");
+
+var LOG_OPTION = {
+    name: "log",
+    description: "Minimum log level to display",
+    values: _.chain(logger.LEVELS).keys().map(stringUtils.toLowerCase).value(),
+    defaults: "info"
 };
 
+var FORMAT_OPTION = {
+    name: "format",
+    description: "Format to build to",
+    values: ["website", "json", "ebook"],
+    defaults: "website"
+};
+
+module.exports = {
+    Book: Book,
+    LOG_LEVELS: logger.LEVELS,
+
+    commands: _.flatten([
+        {
+            name: "build [book] [output]",
+            description: "build a book",
+            options: [
+                FORMAT_OPTION,
+                LOG_OPTION
+            ],
+            exec: function(args, kwargs) {
+                var input = args[0] || process.cwd();
+                var output = args[1] || path.join(input, "_book");
+
+                var book = new Book(input, _.extend({}, {
+                    'config': {
+                        'output': output
+                    },
+                    'logLevel': kwargs.log
+                }));
+
+                return book.parse()
+                .then(function() {
+                    return book.generate(kwargs.format);
+                })
+                .then(function(){
+                    console.log("");
+                    console.log(color.green("Done, without error"));
+                });
+            }
+        },
+
+        _.map(["pdf", "epub", "mobi"], function(ebookType) {
+            return {
+                name: ebookType+" [book] [output]",
+                description: "build a book to "+ebookType,
+                options: [
+                    LOG_OPTION
+                ],
+                exec: function(args, kwargs) {
+                    var input = args[0] || process.cwd();
+                    var output = args[1];
+
+                    var book = new Book(input, _.extend({}, {
+                        'logLevel': kwargs.log
+                    }));
+
+                    return book.parse()
+                    .then(function() {
+                        return book.generateFile(output, {
+                            ebookFormat: ebookType
+                        });
+                    })
+                    .then(function(){
+                        console.log("");
+                        console.log(color.green("Done, without error"));
+                    });
+                }
+            };
+        }),
+
+        {
+            name: "serve [book]",
+            description: "Build then serve a gitbook from a directory",
+            options: [
+                {
+                    name: "port",
+                    description: "Port for server to listen on",
+                    defaults: 4000
+                },
+                {
+                    name: "lrport",
+                    description: "Port for livereload server to listen on",
+                    defaults: 35729
+                },
+                {
+                    name: "watch",
+                    description: "Enable/disable file watcher",
+                    defaults: true
+                },
+                FORMAT_OPTION,
+                LOG_OPTION
+            ],
+            exec: function(args, kwargs) {
+                var input = args[0] || process.cwd();
+                var server = new Server();
+
+                // Init livereload server
+                var lrServer = tinylr({});
+                var lrPath = undefined;
+
+                var generate = function() {
+                    if (server.isRunning()) console.log("Stopping server");
+
+                    return server.stop()
+                    .then(function() {
+                        var book = new Book(input, _.extend({}, {
+                            'config': {
+                                'defaultsPlugins': ["livereload"]
+                            },
+                            'logLevel': kwargs.log
+                        }));
+
+                        return book.parse()
+                        .then(function() {
+                            return book.generate(kwargs.format);
+                        })
+                        .thenResolve(book);
+                    })
+                    .then(function(book) {
+                        console.log();
+                        console.log('Starting server ...');
+                        return server.start(book.options.output, kwargs.port)
+                        .then(function() {
+                            console.log('Serving book on http://localhost:'+kwargs.port);
+
+                            if (lrPath) {
+                                // trigger livereload
+                                lrServer.changed({
+                                    body: {
+                                        files: [lrPath]
+                                    }
+                                });
+                            }
+
+                            if (!kwargs.watch) return;
+
+                            return watch(book.root)
+                            .then(function(filepath) {
+                                // set livereload path
+                                lrPath = filepath;
+                                console.log("Restart after change in files");
+                                console.log('');
+                                return generate();
+                            })
+                        })
+                    });
+                };
+
+                return Q.nfcall(lrServer.listen.bind(lrServer), kwargs.lrport)
+                .then(function() {
+                    console.log('Live reload server started on port:', kwargs.lrport);
+                    console.log('Press CTRL+C to quit ...');
+                    console.log('')
+                    return generate();
+                });
+            }
+        },
+
+        {
+            name: "install [book]",
+            description: "install plugins dependencies",
+            exec: function(args, kwargs) {
+                var input = args[0] || process.cwd();
+
+                var book = new Book(input);
+
+                return book.config.load()
+                .then(function() {
+                    return book.plugins.install();
+                })
+                .then(function(){
+                    console.log("");
+                    console.log(color.green("Done, without error"));
+                });
+            }
+        },
+
+        {
+            name: "init [directory]",
+            description: "create files and folders based on contents of SUMMARY.md",
+            exec: function(args, kwargs) {
+                return Book.init(args[0] || process.cwd())
+                .then(function(){
+                    console.log("");
+                    console.log(color.green("Done, without error"));
+                });
+            }
+        }
+    ])
+};

lib/parse/glossary.js

@@ -1,48 +0,0 @@
-var _ = require('lodash');
-var kramed = require('kramed');
-
-// Get all the pairs of header + paragraph in a list of nodes
-function groups(nodes) {
-    // A list of next nodes
-    var next = nodes.slice(1).concat(null);
-
-    return _.reduce(nodes, function(accu, node, idx) {
-        // Skip
-        if(!(
-            node.type === 'heading' &&
-            (next[idx] && next[idx].type === 'paragraph')
-        )) {
-            return accu;
-        }
-
-        // Add group
-        accu.push([
-            node,
-            next[idx]
-        ]);
-
-        return accu;
-    }, []);
-}
-
-function parseGlossary(src) {
-    var nodes = kramed.lexer(src);
-
-    return groups(nodes)
-    .map(function(pair) {
-        // Simplify each group to a simple object with name/description
-        return {
-            name: pair[0].text,
-            id: entryId(pair[0].text),
-            description: pair[1].text,
-        };
-    });
-}
-
-// Normalizes a glossary entry's name to create an ID
-function entryId(name) {
-    return name.toLowerCase();
-}
-
-module.exports = parseGlossary;
-module.exports.entryId = entryId;

lib/parse/include.js

@@ -1,12 +0,0 @@
-var _ = require('lodash');
-
-module.exports = function(markdown, includer) {
-    // Memoized include function (to cache lookups)
-    var _include = _.memoize(includer);
-
-    return markdown.replace(/{{([\s\S]+?)}}/g, function(match, key) {
-        // If fails leave content as is
-        key = key.trim();
-        return _include(key) || match;
-    });
-};

lib/parse/includer.js

@@ -1,15 +0,0 @@
-// Return a fs inclduer
-module.exports = function(ctx, folders, resolveFile, readFile) {
-    return function(name) {
-        return ctx[name] ||
-        folders.map(function(folder) {
-            // Try including snippet from FS
-            try {
-                var fname = resolveFile(folder, name);
-                // Trim trailing newlines/space of imported snippets
-                return readFile(fname, 'utf8').trimRight();
-            } catch(err) {}
-        })
-        .filter(Boolean)[0];
-    }
-};

lib/parse/index.js

@@ -1,11 +0,0 @@
-module.exports = {
-    summary: require('./summary'),
-    glossary: require('./glossary'),
-    langs: require('./langs'),
-    page: require('./page'),
-    lex: require('./lex'),
-    progress: require('./progress'),
-    navigation: require('./navigation'),
-    readme: require('./readme'),
-    includer: require('./includer')
-};

lib/parse/is_exercise.js

@@ -1,17 +0,0 @@
-var _ = require('lodash');
-
-function isExercise(nodes) {
-    var codeType = { type: 'code' };
-
-    // Number of code nodes in section
-    var len = _.filter(nodes, codeType).length;
-
-    return (
-        // Got 3 or 4 code blocks
-        (len === 3 || len === 4) &&
-        // Ensure all nodes are at the end
-        _.all(_.last(nodes, len), codeType)
-    );
-}
-
-module.exports = isExercise;

lib/parse/is_quiz.js

@@ -1,87 +0,0 @@
-var _ = require('lodash');
-
-function isQuizNode(node) {
-    return (/^[(\[][ x][)\]]/).test(node.text || node);
-}
-
-function isTableQuestion(nodes) {
-    var block = questionBlock(nodes);
-    return (
-        block.length === 1 &&
-        block[0].type === 'table' &&
-        _.all(block[0].cells[0].slice(1), isQuizNode)
-    );
-}
-
-function isListQuestion(nodes) {
-    var block = questionBlock(nodes);
-    // Counter of when we go in and out of lists
-    var inlist = 0;
-    // Number of lists we found
-    var lists = 0;
-    // Elements found outside a list
-    var outsiders = 0;
-    // Ensure that we have nothing except lists
-    _.each(block, function(node) {
-        if(node.type === 'list_start') {
-            inlist++;
-        } else if(node.type === 'list_end') {
-            inlist--;
-            lists++;
-        } else if(inlist === 0) {
-            // Found non list_start or list_end whilst outside a list
-            outsiders++;
-        }
-    });
-    return lists > 0 && outsiders === 0;
-}
-
-function isQuestion(nodes) {
-    return isListQuestion(nodes) || isTableQuestion(nodes);
-}
-
-// Remove (optional) paragraph header node and blockquote
-function questionBlock(nodes) {
-    return nodes.slice(
-        nodes[0].type === 'paragraph' ? 1 : 0,
-        _.findIndex(nodes, { type: 'blockquote_start' })
-    );
-}
-
-function splitQuestions(nodes) {
-    // Represents nodes in current question
-    var buffer = [];
-    return _.reduce(nodes, function(accu, node) {
-        // Add node to buffer
-        buffer.push(node);
-
-        // Flush buffer once we hit the end of a question
-        if(node.type === 'blockquote_end') {
-            accu.push(buffer);
-            // Clear buffer
-            buffer = [];
-        }
-
-        return accu;
-    }, []);
-}
-
-function isQuiz(nodes) {
-    // Extract potential questions
-    var questions = splitQuestions(
-        // Skip quiz title if there
-        nodes.slice(
-            (nodes[0] && nodes[0].type) === 'paragraph' ? 1 : 0
-        )
-    );
-
-    // Nothing that looks like questions
-    if(questions.length === 0) {
-        return false;
-    }
-
-    // Ensure all questions are correctly structured
-    return _.all(questions, isQuestion);
-}
-
-module.exports = isQuiz;

lib/parse/langs.js

@@ -1,25 +0,0 @@
-var _ = require("lodash");
-var parseEntries = require("./summary").entries;
-
-
-var parseLangs = function(content) {
-    var entries = parseEntries(content);
-
-    return {
-        list: _.chain(entries)
-            .filter(function(entry) {
-                return Boolean(entry.path);
-            })
-            .map(function(entry) {
-                return {
-                    title: entry.title,
-                    path: entry.path,
-                    lang: entry.path.replace("/", "")
-                };
-            })
-            .value()
-    };
-};
-
-
-module.exports = parseLangs;

lib/parse/lex.js

@@ -1,79 +0,0 @@
-var _ = require('lodash');
-var kramed = require('kramed');
-
-var isExercise = require('./is_exercise');
-var isQuiz = require('./is_quiz');
-
-// Split a page up into sections (lesson, exercises, ...)
-function splitSections(nodes) {
-    var section = [];
-
-    return _.reduce(nodes, function(sections, el) {
-        if(el.type === 'hr') {
-            sections.push(section);
-            section = [];
-        } else {
-            section.push(el);
-        }
-
-        return sections;
-    }, []).concat([section]); // Add remaining nodes
-}
-
-// What is the type of this section
-function sectionType(nodes, idx) {
-    if(isExercise(nodes)) {
-        return 'exercise';
-    } else if(isQuiz(nodes)) {
-        return 'quiz';
-    }
-
-    return 'normal';
-}
-
-// Generate a uniqueId to identify this section in our code
-function sectionId(section, idx) {
-    return _.uniqueId('gitbook_');
-}
-
-function lexPage(src) {
-    // Lex file
-    var nodes = kramed.lexer(src);
-
-    return _.chain(splitSections(nodes))
-    .map(function(section, idx) {
-        // Detect section type
-        section.type = sectionType(section, idx);
-        return section;
-    })
-    .map(function(section, idx) {
-        // Give each section an ID
-        section.id = sectionId(section, idx);
-        return section;
-
-    })
-    .filter(function(section) {
-        return !_.isEmpty(section);
-    })
-    .reduce(function(sections, section) {
-        var last = _.last(sections);
-
-        // Merge normal sections together
-        if(last && last.type === section.type && last.type === 'normal') {
-            last.push.apply(last, [{'type': 'hr'}].concat(section));
-        } else {
-            // Add to list of sections
-            sections.push(section);
-        }
-
-        return sections;
-    }, [])
-    .map(function(section) {
-        section.links = nodes.links;
-        return section;
-    })
-    .value();
-}
-
-// Exports
-module.exports = lexPage;

lib/parse/page.js

@@ -1,160 +0,0 @@
-var _ = require('lodash');
-var kramed = require('kramed');
-var hljs = require('highlight.js');
-
-var lex = require('./lex');
-var renderer = require('./renderer');
-
-var include = require('./include');
-var lnormalize = require('../utils/lang').normalize;
-
-
-
-// Render a section using our custom renderer
-function render(section, _options) {
-    // Copy section
-    var links = section.links || {};
-    section = _.toArray(section);
-    section.links = links;
-
-    // Build options using defaults and our custom renderer
-    var options = _.extend({}, kramed.defaults, {
-        renderer: renderer(null, _options),
-
-        // Synchronous highlighting with highlight.js
-        highlight: function (code, lang) {
-            if(!lang) return code;
-
-            // Normalize lang
-            lang = lnormalize(lang);
-
-            try {
-                return hljs.highlight(lang, code).value;
-            } catch(e) { }
-
-            return code;
-        }
-    });
-
-    return kramed.parser(section, options);
-}
-
-function quizQuestion(node) {
-    if (node.text) {
-        node.text = node.text.replace(/^([\[(])x([\])])/, "$1 $2");
-    } else {
-        return node.replace(/^([\[(])x([\])])/, "$1 $2");
-    }
-}
-
-function parsePage(src, options) {
-    options = options || {};
-
-    // Lex if not already lexed
-    var parsed = {
-        lexed: (_.isArray(src) ? src : lex(include(src, options.includer || function() { return undefined; })))
-    };
-    parsed.sections = parsed.lexed.map(function(section) {
-        // Transform given type
-        if(section.type === 'exercise') {
-            var nonCodeNodes = _.reject(section, {
-                'type': 'code'
-            });
-
-            var codeNodes = _.filter(section, {
-                'type': 'code'
-            });
-
-            // Languages in code blocks
-            var langs = _.pluck(codeNodes, 'lang').map(lnormalize);
-
-            // Check that they are all the same
-            var validLangs = _.all(_.map(langs, function(lang) {
-                return lang && lang === langs[0];
-            }));
-
-            // Main language
-            var lang = validLangs ? langs[0] : null;
-
-            return {
-                id: section.id,
-                type: section.type,
-                content: render(nonCodeNodes, options),
-                lang: lang,
-                code: {
-                    base: codeNodes[0].text,
-                    solution: codeNodes[1].text,
-                    validation: codeNodes[2].text,
-                    // Context is optional
-                    context: codeNodes[3] ? codeNodes[3].text : null,
-                }
-            };
-        } else if (section.type === 'quiz') {
-            var quiz = [], question, foundFeedback = false;
-            var nonQuizNodes = section[0].type === 'paragraph' && section[1].type !== 'list_start' ? [section[0]] : [];
-            var quizNodes = section.slice(0);
-            quizNodes.splice(0, nonQuizNodes.length);
-
-            for (var i = 0; i < quizNodes.length; i++) {
-                var node = quizNodes[i];
-
-                if (question && (((node.type === 'list_end' || node.type === 'blockquote_end') && i === quizNodes.length - 1)
-                                 || node.type === 'table' || (node.type === 'paragraph' && !foundFeedback))) {
-                    quiz.push({
-                        base: render(question.questionNodes, options),
-                        solution: render(question.solutionNodes, options),
-                        feedback: render(question.feedbackNodes, options)
-                    });
-                }
-
-                if (node.type === 'table' || (node.type === 'paragraph' && !foundFeedback)) {
-                    question = { questionNodes: [], solutionNodes: [], feedbackNodes: [] };
-                }
-
-                if (node.type === 'blockquote_start') {
-                    foundFeedback = true;
-                } else if (node.type === 'blockquote_end') {
-                    foundFeedback = false;
-                }
-
-                if (node.type === 'table') {
-                    question.solutionNodes.push(_.cloneDeep(node));
-                    node.cells = node.cells.map(function(row) {
-                        return row.map(quizQuestion);
-                    });
-                    question.questionNodes.push(node);
-                } else if (!/blockquote/.test(node.type)) {
-                    if (foundFeedback) {
-                        question.feedbackNodes.push(node);
-                    } else if (node.type === 'paragraph' || node.type === 'text'){
-                        question.solutionNodes.push(_.cloneDeep(node));
-                        quizQuestion(node);
-                        question.questionNodes.push(node);
-                    } else {
-                        question.solutionNodes.push(node);
-                        question.questionNodes.push(node);
-                    }
-                }
-            }
-
-            return {
-                id: section.id,
-                type: section.type,
-                content: render(nonQuizNodes, options),
-                quiz: quiz
-            };
-        }
-
-        // Render normal pages
-        return {
-            id: section.id,
-            type: section.type,
-            content: render(section, options)
-        };
-    });
-
-    return parsed;
-}
-
-// Exports
-module.exports = parsePage;

lib/parse/readme.js

@@ -1,45 +0,0 @@
-var _ = require('lodash');
-var kramed = require('kramed');
-var textRenderer = require('kramed-text-renderer');
-
-function extractFirstNode(nodes, nType) {
-    return _.chain(nodes)
-    .filter(function(node) {
-        return node.type == nType;
-    })
-    .pluck("text")
-    .first()
-    .value();
-}
-
-
-function parseReadme(src) {
-    var nodes, title, description;
-    var renderer = textRenderer();
-
-    // Parse content
-    nodes = kramed.lexer(src);
-
-    title = extractFirstNode(nodes, "heading") || '';
-    description = extractFirstNode(nodes, "paragraph") || '';
-
-    var convert = _.compose(
-        function(text) {
-            return _.unescape(text.replace(/(\r\n|\n|\r)/gm, ""));
-        },
-        function(text) {
-            return kramed.parse(text, _.extend({}, kramed.defaults, {
-                renderer: renderer
-            }));
-        }
-    );
-
-    return {
-        title: convert(title),
-        description: convert(description)
-    };
-}
-
-
-// Exports
-module.exports = parseReadme;

lib/parse/renderer.js

@@ -1,141 +0,0 @@
-var url = require('url');
-var _ = require('lodash');
-var inherits = require('util').inherits;
-var links = require('../utils').links;
-var kramed = require('kramed');
-
-var rendererId = 0;
-
-function GitBookRenderer(options, extra_options) {
-    if(!(this instanceof GitBookRenderer)) {
-        return new GitBookRenderer(options, extra_options);
-    }
-    GitBookRenderer.super_.call(this, options);
-
-    this._extra_options = extra_options;
-    this.quizRowId = 0;
-    this.id = rendererId++;
-    this.quizIndex = 0;
-}
-inherits(GitBookRenderer, kramed.Renderer);
-
-GitBookRenderer.prototype._unsanitized = function(href) {
-    var prot = '';
-    try {
-        prot = decodeURIComponent(unescape(href))
-            .replace(/[^\w:]/g, '')
-            .toLowerCase();
-
-    } catch (e) {
-        return true;
-    }
-
-    if(prot.indexOf('javascript:') === 0) {
-        return true;
-    }
-
-    return false;
-};
-
-GitBookRenderer.prototype.link = function(href, title, text) {
-    // Our "fixed" href
-    var _href = href;
-
-    // Don't build if it looks malicious
-    if (this.options.sanitize && this._unsanitized(href)) {
-        return text;
-    }
-
-    // Parsed version of the url
-    var parsed = url.parse(href);
-    var o = this._extra_options;
-    var extname = parsed.path? _.last(parsed.path.split(".")) : "";
-
-    // Relative link, rewrite it to point to github repo
-    if(links.isRelative(_href) && extname == "md") {
-        _href = links.toAbsolute(_href, o.dir || "./", o.outdir || "./");
-        _href = _href.replace(".md", ".html");
-    }
-
-    // Generate HTML for link
-    var out = '<a href="' + _href + '"';
-    // Title if no null
-    if (title) {
-        out += ' title="' + title + '"';
-    }
-    // Target blank if external
-    if(parsed.protocol) {
-        out += ' target="_blank"';
-    }
-    out += '>' + text + '</a>';
-    return out;
-};
-
-GitBookRenderer.prototype.image = function(href, title, text) {
-    // Our "fixed" href
-    var _href = href;
-
-    // Parsed version of the url
-    var parsed = url.parse(href);
-
-    // Options
-    var o = this._extra_options;
-
-    // Relative image, rewrite it depending output
-    if(links.isRelative(href) && o && o.dir && o.outdir) {
-        // o.dir: directory parent of the file currently in rendering process
-        // o.outdir: directory parent from the html output
-
-        _href = links.toAbsolute(_href, o.dir, o.outdir);
-    }
-
-    return GitBookRenderer.super_.prototype.image.call(this, _href, title, text);
-};
-
-GitBookRenderer.prototype.tablerow = function(content) {
-    this.quizRowId += 1;
-    return GitBookRenderer.super_.prototype.tablerow(content);
-};
-
-var fieldRegex = /^([(\[])([ x])[\])]/;
-GitBookRenderer.prototype._createCheckboxAndRadios = function(text) {
-    var match = fieldRegex.exec(text);
-    if (!match) {
-        return text;
-    }
-    //fix radio input uncheck failed
-    var quizFieldName='quiz-row-' + this.id + '-' + this.quizRowId ;
-    var quizIdentifier = quizFieldName + '-' + this.quizIndex++;
-    var field = "<input name='" + quizFieldName + "' id='" + quizIdentifier + "' type='";
-    field += match[1] === '(' ? "radio" : "checkbox";
-    field += match[2] === 'x' ? "' checked/>" : "'/>";
-    var splittedText = text.split(fieldRegex);
-    var length = splittedText.length;
-    var label = '<label class="quiz-label" for="' + quizIdentifier + '">' + splittedText[length - 1] + '</label>';
-    return text.replace(fieldRegex, field).replace(splittedText[length - 1], label);
-};
-
-GitBookRenderer.prototype.tablecell = function(content, flags) {
-    return GitBookRenderer.super_.prototype.tablecell(this._createCheckboxAndRadios(content), flags);
-};
-
-GitBookRenderer.prototype.listitem = function(text) {
-    return GitBookRenderer.super_.prototype.listitem(this._createCheckboxAndRadios(text));
-};
-
-GitBookRenderer.prototype.code = function(code, lang, escaped) {
-    return GitBookRenderer.super_.prototype.code.call(
-        this,
-        code,
-        lang,
-        escaped
-    );
-};
-
-GitBookRenderer.prototype.heading = function(text, level, raw) {
-    var id = this.options.headerPrefix + raw.toLowerCase().replace(/[^\w -]+/g, '').replace(/ /g, '-');
-    return '<h' + level + ' id="' + id + '">' + text + '</h' + level + '>\n';
-};
-
-// Exports
-module.exports = GitBookRenderer;

lib/parse/summary.js

@@ -1,148 +0,0 @@
-var _ = require('lodash');
-var kramed = require('kramed');
-
-
-// Utility function for splitting a list into groups
-function splitBy(list, starter, ender) {
-    var starts = 0;
-    var ends = 0;
-    var group = [];
-
-    // Groups
-    return _.reduce(list, function(groups, value) {
-        // Ignore start and end delimiters in resulted groups
-        if(starter(value)) {
-            starts++;
-        } else if(ender(value)) {
-            ends++;
-        }
-
-        // Add current value to group
-        group.push(value);
-
-        // We've got a matching
-        if(starts === ends && starts !== 0) {
-            // Add group to end groups
-            // (remove starter and ender token)
-            groups.push(group.slice(1, -1));
-
-            // Reset group
-            group = [];
-        }
-
-        return groups;
-    }, []);
-}
-
-function listSplit(nodes, start_type, end_type) {
-    return splitBy(nodes, function(el) {
-        return el.type === start_type;
-    }, function(el) {
-        return el.type === end_type;
-    });
-}
-
-// Get the biggest list
-// out of a list of kramed nodes
-function filterList(nodes) {
-    return _.chain(nodes)
-    .toArray()
-    .rest(function(el) {
-        // Get everything after list_start
-        return el.type !== 'list_start';
-    })
-    .reverse()
-    .rest(function(el) {
-        // Get everything after list_end (remember we're reversed)
-        return el.type !== 'list_end';
-    })
-    .reverse()
-    .value().slice(1, -1);
-}
-
-// Parses an Article or Chapter title
-// supports extracting links
-function parseTitle(src, nums) {
-    // Check if it's a link
-    var matches = kramed.InlineLexer.rules.link.exec(src);
-
-    var level = nums.join('.');
-
-    // Not a link, return plain text
-    if(!matches) {
-        return {
-            title: src,
-            level: level,
-            path: null,
-        };
-    }
-
-    return {
-        title: matches[1],
-        level: level,
-
-        // Normalize path
-        // 1. Convert Window's "\" to "/"
-        // 2. Remove leading "/" if exists
-        path: matches[2].replace(/\\/g, '/').replace(/^\/+/, ''),
-    };
-}
-
-function parseChapter(nodes, nums) {
-    // Convert single number to an array
-    nums = _.isArray(nums) ? nums : [nums];
-
-    return _.extend(parseTitle(_.first(nodes).text, nums), {
-        articles: _.map(listSplit(filterList(nodes), 'list_item_start', 'list_item_end'), function(nodes, i) {
-            return parseChapter(nodes, nums.concat(i + 1));
-        })
-    });
-}
-
-function defaultChapterList(chapterList) {
-    var first = _.first(chapterList);
-
-    // Check if introduction node was specified in SUMMARY.md
-    if (first) {
-        var chapter = parseChapter(first, [0]);
-
-        // Already have README node, we're good to go
-        if(chapter.path === 'README.md') {
-            return chapterList;
-        }
-    }
-
-    // It wasn't specified, so add in default
-    return [
-        [ { type: 'text', text: '[Introduction](README.md)' } ]
-    ].concat(chapterList);
-}
-
-function listGroups(src) {
-    var nodes = kramed.lexer(src);
-
-    // Get out groups of lists
-    return listSplit(
-        filterList(nodes),
-        'list_item_start', 'list_item_end'
-    );
-}
-
-function parseSummary(src) {
-    // Split out chapter sections
-    var chapters = defaultChapterList(listGroups(src))
-    .map(parseChapter);
-
-    return {
-        chapters: chapters
-    };
-}
-
-function parseEntries (src) {
-    return listGroups(src).map(parseChapter);
-}
-
-
-// Exports
-module.exports = parseSummary;
-module.exports.entries = parseEntries;

lib/plugin.js

@@ -0,0 +1,183 @@
+var _ = require("lodash");
+var Q = require("q");
+var semver = require("semver");
+var path = require("path");
+var url = require("url");
+var fs = require("./utils/fs");
+var resolve = require('resolve');
+
+var pkg = require("../package.json");
+
+var Plugin = function(book, name) {
+    this.book = book;
+    this.name = name;
+    this.packageInfos = {};
+    this.infos = {};
+
+    // Bind methods
+    _.bindAll(this);
+
+    _.each([
+        "gitbook-plugin-"+name,
+        "gitbook-"+name,
+        name,
+    ], function(_name) {
+        // Load from the book
+        if (this.load(_name, book.root)) return false;
+
+        // Load from default plugins
+        if (this.load(_name, __dirname)) return false;
+    }, this);
+};
+
+// Type of plugins resources
+Plugin.RESOURCES = ["js", "css"];
+Plugin.HOOKS = [
+    "init", "finish", "finish:before"
+]
+
+// Load from a name
+Plugin.prototype.load = function(name, baseDir) {
+    try {
+        var res = resolve.sync(name+"/package.json", { basedir: baseDir });
+
+        this.baseDir = path.dirname(res);
+        this.packageInfos = require(res);
+        this.infos = require(resolve.sync(name, { basedir: baseDir }));
+        this.name = name;
+
+        return true;
+    } catch (e) {
+        this.packageInfos = {};
+        this.infos = {};
+        return false;
+    }
+};
+
+Plugin.prototype.normalizeResource = function(resource) {
+    // Parse the resource path
+    var parsed = url.parse(resource);
+
+    // This is a remote resource
+    // so we will simply link to using it's URL
+    if (parsed.protocol) {
+        return {
+            "url": resource
+        };
+    }
+
+    // This will be copied over from disk
+    // and shipped with the book's build
+    return { "path": this.name+"/"+resource };
+};
+
+// Return resources
+Plugin.prototype._getResources = function(base) {
+    base = base;
+    var book = this.infos[base];
+
+    // Compatibility with version 1.x.x
+    if (base == "website") book = book || this.infos["book"];
+
+    // Nothing specified, fallback to default
+    if (!book) {
+        return Q({});
+    }
+
+    // Dynamic function
+    if(typeof book === "function") {
+        // Call giving it the context of our book
+        return Q().then(book.bind(this.book));
+    }
+
+    // Plain data object
+    return Q(_.cloneDeep(book));
+};
+
+// Normalize resources and return them
+Plugin.prototype.getResources = function(base) {
+    var that = this;
+
+    return this._getResources(base)
+    .then(function(resources) {
+
+        _.each(Plugin.RESOURCES, function(resourceType) {
+            resources[resourceType] = (resources[resourceType] || []).map(that.normalizeResource);
+        });
+
+        return resources;
+    });
+};
+
+// Normalize filters and return them
+Plugin.prototype.getFilters = function() {
+    return this.infos.filters || {};
+};
+
+// Normalize blocks and return them
+Plugin.prototype.getBlocks = function() {
+    return this.infos.blocks || {};
+};
+
+// Test if it's a valid plugin
+Plugin.prototype.isValid = function() {
+    var that = this;
+    var isValid = (
+        this.packageInfos &&
+        this.packageInfos.name &&
+        this.packageInfos.engines &&
+        this.packageInfos.engines.gitbook &&
+        semver.satisfies(pkg.version, this.packageInfos.engines.gitbook)
+    );
+
+    // Valid hooks
+    _.each(this.infos.hooks, function(hook, hookName) {
+        if (_.contains(Plugin.HOOKS, hookName)) return;
+        that.book.log.warn.ln("Hook '"+hookName+" 'used by plugin '"+that.packageInfos.name+"' has been removed or is deprecated");
+    });
+
+    return isValid;
+};
+
+// Resolve file path
+Plugin.prototype.resolveFile = function(filename) {
+    return path.resolve(this.baseDir, filename);
+};
+
+// Resolve file path
+Plugin.prototype.callHook = function(name, data) {
+    // Our book will be the context to apply
+    var context = this.book;
+
+    var hookFunc = this.infos.hooks? this.infos.hooks[name] : null;
+    data = data || {};
+
+    if (!hookFunc) return Q(data);
+
+    this.book.log.debug.ln("call hook", name);
+    if (!_.contains(Plugin.HOOKS, name)) this.book.log.warn.ln("hook '"+name+"' used by plugin '"+this.name+"' is deprecated, and will be remove in the coming versions");
+
+    return Q()
+    .then(function() {
+        return hookFunc.apply(context, [data]);
+    });
+};
+
+// Copy plugin assets fodler
+Plugin.prototype.copyAssets = function(out, base) {
+    var that = this;
+
+    return this.getResources(base)
+    .get('assets')
+    .then(function(assets) {
+        // Assets are undefined
+        if(!assets) return false;
+
+        return fs.copy(
+            that.resolveFile(assets),
+            out
+        ).then(_.constant(true));
+    }, _.constant(false));
+};
+
+module.exports = Plugin;

lib/pluginslist.js

@@ -0,0 +1,225 @@
+var _ = require("lodash");
+var Q = require("q");
+var npmi = require('npmi');
+var npm = require('npm');
+var semver = require('semver');
+
+var Plugin = require("./plugin");
+var pkg = require("../package.json");
+
+var initNPM = _.memoize(function() {
+    return Q.nfcall(npm.load, { silent: true, loglevel: 'silent' });
+});
+
+
+var PluginsList = function(book, plugins) {
+    this.book = book;
+    this.log = this.book.log;
+
+    // List of Plugin objects
+    this.list = [];
+
+    // List of names of failed plugins
+    this.failed = [];
+
+    // Namespaces
+    this.namespaces = _.chain(["website", "ebook"])
+        .map(function(namespace) {
+            return [
+                namespace,
+                {
+                    html: {},
+                    resources: _.chain(Plugin.RESOURCES)
+                        .map(function(type) {
+                            return [type, []];
+                        })
+                        .object()
+                        .value()
+                }
+            ];
+        })
+        .object()
+        .value();
+
+    // Bind methods
+    _.bindAll(this);
+
+    if (plugins) this.load(plugins);
+};
+
+// return count of plugins
+PluginsList.prototype.count = function() {
+    return this.list.length;
+};
+
+// Add and load a plugin
+PluginsList.prototype.load = function(plugin, options) {
+    var that = this;
+
+    if (_.isArray(plugin)) {
+        return _.reduce(plugin, function(prev, p) {
+            return prev.then(function() {
+                return that.load(p);
+            });
+        }, Q());
+    }
+    if (_.isObject(plugin) && !(plugin instanceof Plugin)) plugin = plugin.name;
+    if (_.isString(plugin)) plugin = new Plugin(this.book, plugin);
+
+    that.log.info("load plugin", plugin.name, "....");
+    if (!plugin.isValid()) {
+        that.log.info.fail();
+        that.failed.push(plugin.name);
+        return Q();
+    } else {
+        that.log.info.ok();
+
+        // Push in the list
+        that.list.push(plugin);
+    }
+
+    // Extract filters
+    _.each(plugin.getFilters(), function(filterFunc, filterName) {
+        that.book.template.addFilter(filterName, filterFunc);
+    });
+
+    // Extract blocks
+    _.each(plugin.getBlocks(), function(block, blockName) {
+        that.book.template.addBlock(blockName, block);
+    });
+
+    return _.reduce(_.keys(that.namespaces), function(prev, namespaceName) {
+        return prev.then(function() {
+            return plugin.getResources(namespaceName)
+            .then(function(plResources) {
+                var namespace = that.namespaces[namespaceName];
+
+                // Extract js and css
+                _.each(Plugin.RESOURCES, function(resourceType) {
+                    namespace.resources[resourceType] = (namespace.resources[resourceType] || []).concat(plResources[resourceType] || []);
+                });
+
+                // Map of html resources by name added by each plugin
+                _.each(plResources.html || {}, function(value, tag) {
+                    // Turn into function if not one already
+                    if (!_.isFunction(value)) value = _.constant(value);
+
+                    namespace.html[tag] = namespace.html[tag] || [];
+                    namespace.html[tag].push(value);
+                });
+            })
+        });
+    }, Q());
+};
+
+// Call a hook
+PluginsList.prototype.hook = function(name, data) {
+    return _.reduce(this.list, function(prev, plugin) {
+        return prev.then(function(ret) {
+            return plugin.callHook(name, ret);
+        });
+    }, Q(data));
+};
+
+// Return a template from a plugin
+PluginsList.prototype.template = function(name) {
+    var withTpl = _.find(this.list, function(plugin) {
+        return (
+            plugin.infos.templates &&
+            plugin.infos.templates[name]
+        );
+    });
+
+    if (!withTpl) return null;
+    return withTpl.resolveFile(withTpl.infos.templates[name]);
+};
+
+// Return an html snippet
+PluginsList.prototype.html = function(namespace, tag, context, options) {
+    var htmlSnippets = this.namespaces[namespace].html[tag];
+    return _.map(htmlSnippets || [], function(code) {
+        return code.call(context, options);
+    }).join("\n");
+};
+
+// Return a resources map for a namespace
+PluginsList.prototype.resources = function(namespace) {
+    return this.namespaces[namespace].resources;
+};
+
+// Install plugins from a book
+PluginsList.prototype.install = function() {
+    var that = this;
+    var defaultsPlugins = _.pluck(that.book.options.defaultsPlugins)
+
+    // Remove defaults (no need to install)
+    var plugins = _.filter(that.book.options.plugins, function(plugin) {
+        return !_.contains(defaultsPlugins, plugin.name);
+    });
+
+    // Install plugins one by one
+    that.book.log.info.ln(plugins.length+" plugins to install");
+    return _.reduce(plugins, function(prev, plugin) {
+        return prev.then(function() {
+            var fullname = "gitbook-plugin-"+plugin.name;
+
+            return Q()
+
+            // Resolve version if needed
+            .then(function() {
+                if (plugin.version) return plugin.version;
+
+                that.book.log.info.ln("No version specified, resolve plugin", plugin.name);
+                return initNPM()
+                .then(function() {
+                    return Q.nfcall(npm.commands.view, [fullname+"@*", "engines"], true);
+                })
+                .then(function(versions) {
+                    return _.chain(versions)
+                        .pairs()
+                        .map(function(v) {
+                            return {
+                                version: v[0],
+                                gitbook: (v[1].engines || {})["gitbook"]
+                            }
+                        })
+                        .filter(function(v) {
+                            return v.gitbook && semver.satisfies(pkg.version, v.gitbook);
+                        })
+                        .sort(function(v1, v2) {
+                            return semver.lt(v1.version, v2.version)? 1 : -1;
+                        })
+                        .pluck("version")
+                        .first()
+                        .value();
+                });
+            })
+
+            // Install the plugin with the resolved version
+            .then(function(version) {
+                if (!version) {
+                    throw "Found no satisfactory version for plugin "+plugin.name;
+                }
+
+                that.book.log.info.ln("install plugin", plugin.name, "from npm ("+fullname+") with version", version);
+                return Q.nfcall(npmi, {
+                    'name': fullname,
+                    'version': version,
+                    'path': that.book.root,
+                    'npmLoad': {
+                        'loglevel': 'silent',
+                        'loaded': true,
+                        'prefix': that.book.root
+                    }
+                });
+            })
+            .then(function() {
+                that.book.log.info.ok("plugin", plugin.name, "installed with success");
+            });
+        });
+    }, Q());
+};
+
+
+
+module.exports = PluginsList;

lib/template.js

@@ -0,0 +1,394 @@
+var _ = require("lodash");
+var Q = require("q");
+var path = require("path");
+var nunjucks = require("nunjucks");
+
+var git = require("./utils/git");
+var stringUtils = require("./utils/string");
+var fs = require("./utils/fs");
+var batch = require("./utils/batch");
+var pkg = require("../package.json");
+
+
+// The loader should handle relative and git url
+var BookLoader = nunjucks.Loader.extend({
+    async: true,
+
+    init: function(book) {
+        this.book = book;
+    },
+
+    getSource: function(fileurl, callback) {
+        var that = this;
+
+        git.resolveFile(fileurl)
+        .then(function(filepath) {
+            // Is local file
+            if (!filepath) filepath = path.resolve(that.book.root, fileurl);
+            else that.book.log.debug.ln("resolve from git", fileurl, "to", filepath)
+
+            //  Read file from absolute path
+            return fs.readFile(filepath)
+            .then(function(source) {
+                return {
+                    src: source.toString(),
+                    path: filepath
+                }
+            });
+        })
+        .nodeify(callback);
+    },
+
+    resolve: function(from, to) {
+        return path.resolve(path.dirname(from), to);
+    }
+});
+
+
+var TemplateEngine = function(book) {
+    this.book = book;
+    this.log = this.book.log;
+
+    // Nunjucks env
+    this.env = new nunjucks.Environment(
+        new BookLoader(book),
+        {
+            // Escaping is done after by the markdown parser
+            autoescape: false,
+
+            // Tags
+            tags: {
+                blockStart: '{%',
+                blockEnd: '%}',
+                variableStart: '{{',
+                variableEnd: '}}',
+                commentStart: '{###',
+                commentEnd: '###}'
+            }
+        }
+    );
+
+    // List of tags shortcuts
+    this.shortcuts = [];
+
+    // Map of blocks
+    this.blocks = {};
+
+    // Bind methods
+    _.bindAll(this);
+
+    // Default block "html" that return html not parsed
+    this.addBlock("html", {
+        process: _.identity
+    });
+};
+
+// Process a block in a context
+TemplateEngine.prototype.processBlock = function(blk) {
+    if (_.isString(blk)) blk = { body: blk };
+
+    blk = _.defaults(blk, {
+        parse: false,
+        post: undefined
+    });
+    blk.id = _.uniqueId("blk");
+
+    var toAdd = (!blk.parse) || (blk.post != undefined);
+
+    // Add to global map
+    if (toAdd) this.blocks[blk.id] = blk;
+
+    //Parsable block, just return it
+    if (blk.parse) {
+        return blk.body;
+    }
+
+    // Return it as a macro
+    return "%+%"+blk.id+"%+%";
+};
+
+// Replace blocks by body after processing
+// This is done to avoid that markdown processer parse the block content
+TemplateEngine.prototype.replaceBlocks = function(content) {
+    var that = this;
+
+    return content.replace(/\%\+\%([\s\S]+?)\%\+\%/g, function(match, key) {
+        var blk = that.blocks[key];
+        if (!blk) return match;
+
+        var body = blk.body;
+
+        return body;
+    });
+};
+
+// Bind a function to a context
+TemplateEngine.prototype.bindContext = function(func) {
+    var that = this;
+
+    return function() {
+        var ctx = {
+            ctx: this.ctx,
+            book: that.book,
+            generator: that.book.options.generator
+        };
+
+        return func.apply(ctx, arguments);
+    };
+};
+
+// Add filter
+TemplateEngine.prototype.addFilter = function(filterName, func) {
+    try {
+        this.env.getFilter(filterName);
+        this.log.warn.ln("conflict in filters, '"+filterName+"' is already set");
+        return false;
+    } catch(e) {}
+
+    this.log.debug.ln("add filter '"+filterName+"'");
+    this.env.addFilter(filterName, this.bindContext(function() {
+        var ctx = this;
+        var args = Array.prototype.slice.apply(arguments);
+        var callback = _.last(args);
+
+        Q()
+        .then(function() {
+            return func.apply(ctx, args.slice(0, -1));
+        })
+        .nodeify(callback);
+    }), true);
+    return true;
+};
+
+// Add a block
+TemplateEngine.prototype.addBlock = function(name, block) {
+    var that = this;
+
+    block = _.defaults(block || {}, {
+        shortcuts: [],
+        end: "end"+name,
+        process: _.identity,
+        blocks: []
+    });
+
+    var extName = 'Block'+name+'Extension';
+    if (this.env.getExtension(extName)) {
+        this.log.warn.ln("conflict in blocks, '"+name+"' is already defined");
+        return false;
+    }
+
+    this.log.debug.ln("add block '"+name+"'");
+
+    var Ext = function () {
+        this.tags = [name];
+
+        this.parse = function(parser, nodes, lexer) {
+            var body = null;
+            var lastBlockName = null;
+            var lastBlockArgs = null;
+            var allBlocks = block.blocks.concat([block.end]);
+            var subbodies = {};
+
+            var tok = parser.nextToken();
+            var args = parser.parseSignature(null, true);
+            parser.advanceAfterBlockEnd(tok.value);
+
+            while (1) {
+                // Read body
+                var currentBody = parser.parseUntilBlocks.apply(parser, allBlocks);
+
+                // Handle body with previous block name and args
+                if (lastBlockName) {
+                    subbodies[lastBlockName] = subbodies[lastBlockName] || [];
+                    subbodies[lastBlockName].push({
+                        body: currentBody,
+                        args: lastBlockArgs
+                    });
+                } else {
+                    body = currentBody;
+                }
+
+                // Read new block
+                lastBlockName = parser.peekToken().value;
+                if (lastBlockName == block.end) {
+                    break;
+                }
+
+                // Parse signature and move to the end of the block
+                lastBlockArgs = parser.parseSignature(null, true);
+                parser.advanceAfterBlockEnd(lastBlockName);
+            }
+            parser.advanceAfterBlockEnd();
+
+            var bodies = [body];
+            _.each(block.blocks, function(blockName) {
+                subbodies[blockName] = subbodies[blockName] || [];
+                if (subbodies[blockName].length == 0) {
+                    subbodies[blockName].push({
+                        args: new nodes.NodeList(),
+                        body: new nodes.NodeList()
+                    });
+                }
+
+                bodies.push(subbodies[blockName][0].body);
+            });
+
+            return new nodes.CallExtensionAsync(this, 'run', args, bodies);
+        };
+
+        this.run = function(context) {
+            var args = Array.prototype.slice.call(arguments, 1);
+            var callback = args.pop();
+
+            // Extract blocks
+            var blocks = args
+                .concat([])
+                .slice(-block.blocks.length);
+
+            // Eliminate blocks from list
+            if (block.blocks.length > 0) args = args.slice(0, -block.blocks.length);
+
+            // Extract main body and kwargs
+            var body = args.pop();
+            var kwargs = args.pop() || {};
+
+            // Extract blocks body
+            var _blocks =  _.map(block.blocks, function(blockName, i){
+                return {
+                    name: blockName,
+                    body: blocks[i]()
+                };
+            });
+
+            var func = that.bindContext(block.process);
+
+            Q()
+            .then(function() {
+                return func.call(context, {
+                    body: body(),
+                    args: args,
+                    kwargs: kwargs,
+                    blocks: _blocks
+                });
+            })
+
+            // process the block returned
+            .then(that.processBlock)
+            .nodeify(callback)
+        };
+    };
+
+
+    // Add the Extension
+    this.env.addExtension(extName, new Ext());
+
+    // Add shortcuts
+    if (!_.isArray(block.shortcuts)) block.shortcuts = [block.shortcuts];
+    _.each(block.shortcuts, function(shortcut) {
+        this.log.debug.ln("add template shortcut from '"+shortcut.start+"' to block '"+name+"' for parsers ", shortcut.parsers);
+        this.shortcuts.push({
+            parsers: shortcut.parsers,
+            start: shortcut.start,
+            end: shortcut.end,
+            tag: {
+                start: name,
+                end: block.end
+            }
+        });
+    }, this);
+};
+
+// Apply a shortcut to a string
+TemplateEngine.prototype._applyShortcut = function(parser, content, shortcut) {
+    if (!_.contains(shortcut.parsers, parser)) return content;
+    var regex = new RegExp(
+        stringUtils.escapeRegex(shortcut.start) + "([\\s\\S]*?[^\\$])" + stringUtils.escapeRegex(shortcut.end),
+       'g'
+    );
+    return content.replace(regex, function(all, match) {
+        return "{% "+shortcut.tag.start+" %}"+ match + "{% "+shortcut.tag.end+" %}";
+    });
+};
+
+// Render a string from the book
+TemplateEngine.prototype.renderString = function(content, context, options) {
+    var context = _.extend({}, context, {
+        // Variables from book.json
+        book: this.book.options.variables,
+
+        // infos about gitbook
+        gitbook: {
+            version: pkg.version,
+            generator: this.book.options.generator
+        }
+    });
+    options = _.defaults(options || {}, {
+        path: null,
+        type: null
+    });
+    if (options.path) options.path = this.book.resolve(options.path);
+
+    // Replace shortcuts
+    content = _.reduce(this.shortcuts, _.partial(this._applyShortcut.bind(this), options.type), content);
+
+    return Q.nfcall(this.env.renderString.bind(this.env), content, context, options);
+};
+
+// Render a file from the book
+TemplateEngine.prototype.renderFile = function(filename, options) {
+    var that = this, context;
+
+    return that.book.readFile(filename)
+    .then(function(content) {
+        return that.renderString(content, {}, {
+            path: filename
+        });
+    });
+};
+
+// Render a page from the book
+TemplateEngine.prototype.renderPage = function(page) {
+    var that = this, context;
+
+    return that.book.statFile(page.path)
+    .then(function(stat) {
+       context = {
+            // infos about the file
+            file: {
+                path: page.path,
+                mtime: stat.mtime
+            }
+        };
+
+        return that.renderString(page.content, context, {
+            path: page.path,
+            type: page.type
+        });
+    });
+};
+
+// Post process content
+TemplateEngine.prototype.postProcess = function(content) {
+    var that = this;
+
+    return Q(content)
+    .then(that.replaceBlocks)
+    .then(function(_content) {
+        return batch.execEach(that.blocks, {
+            max: 20,
+            fn: function(blk, blkId) {
+                return Q()
+                .then(function() {
+                    if (!blk.post) return Q();
+                    return blk.post();
+                })
+                .then(function() {
+                    delete that.blocks[blkId];
+                });
+            }
+        })
+        .thenResolve(_content);
+    });
+};
+
+module.exports = TemplateEngine;

lib/utils/batch.js

@@ -0,0 +1,52 @@
+var Q = require("q");
+var _ = require("lodash");
+
+// Execute a method for all element
+function execEach(items, options) {
+    if (_.size(items) == 0) return Q();
+    var concurrents = 0, d = Q.defer(), pending = [];
+
+    options = _.defaults(options || {}, {
+        max: 100,
+        fn: function(item) {}
+    });
+
+
+    function startItem(item, i) {
+        if (concurrents >= options.max) {
+            pending.push([item, i]);
+            return;
+        }
+
+        concurrents++;
+        Q()
+        .then(function() {
+            return options.fn(item, i);
+        })
+        .then(function() {
+            concurrents--;
+
+            // Next pending
+            var next = pending.shift();
+
+            if (concurrents == 0 && !next) {
+                d.resolve();
+            } else if (next) {
+                startItem.apply(null, next);
+            }
+        })
+        .fail(function(err) {
+            pending = [];
+            d.reject(err);
+        })
+    }
+
+    _.each(items, startItem);
+
+    return d.promise;
+}
+
+module.exports = {
+    execEach: execEach
+};
+

lib/utils/code.js

@@ -1,6 +1,9 @@
+var hljs = require('highlight.js');
+
 var MAP = {
     'py': 'python',
     'js': 'javascript',
+    'json': 'javascript',
     'rb': 'ruby',
     'csharp': 'cs',
 };
@@ -12,8 +15,22 @@ function normalize(lang) {
     return MAP[lower] || lower;
 }
 
+function highlight(lang, code) {
+    if(!lang) return code;
+
+    // Normalize lang
+    lang = normalize(lang);
+
+    try {
+        return hljs.highlight(lang, code).value;
+    } catch(e) { }
+
+    return code;
+}
+
 // Exports
 module.exports = {
+    highlight: highlight,
     normalize: normalize,
     MAP: MAP
 };

lib/utils/fs.js

@@ -0,0 +1,178 @@
+var _ = require("lodash");
+var Q = require("q");
+var tmp = require("tmp");
+var path = require("path");
+var fs = require("graceful-fs");
+var fsExtra = require("fs-extra");
+var Ignore = require("fstream-ignore");
+
+var fsUtils = {
+    tmp: {
+        file: function(opt) {
+            return Q.nfcall(tmp.file.bind(tmp), opt).get(0)
+        },
+        dir: function() {
+            return Q.nfcall(tmp.dir.bind(tmp)).get(0)
+        }
+    },
+    list: listFiles,
+    stat: Q.denodeify(fs.stat),
+    readdir: Q.denodeify(fs.readdir),
+    readFile: Q.denodeify(fs.readFile),
+    writeFile: writeFile,
+    writeStream: writeStream,
+    mkdirp: Q.denodeify(fsExtra.mkdirp),
+    copy: Q.denodeify(fsExtra.copy),
+    remove: Q.denodeify(fsExtra.remove),
+    symlink: Q.denodeify(fsExtra.symlink),
+    exists: function(path) {
+        var d = Q.defer();
+        fs.exists(path, d.resolve);
+        return d.promise;
+    },
+    existsSync: fs.existsSync.bind(fs),
+    readFileSync: fs.readFileSync.bind(fs),
+    clean: cleanFolder,
+    getUniqueFilename: getUniqueFilename,
+}
+
+// Write a file
+function writeFile(filename, data, options) {
+    var d = Q.defer();
+
+    try {
+        fs.writeFileSync(filename, data, options)
+    } catch(err) {
+        d.reject(err);
+    }
+    d.resolve();
+
+
+    return d.promise;
+}
+
+// Write a stream to a file
+function writeStream(filename, st) {
+    var d = Q.defer();
+
+    var wstream = fs.createWriteStream(filename);
+
+    wstream.on('finish', function () {
+        d.resolve();
+    });
+    wstream.on('error', function (err) {
+        d.reject(err);
+    });
+
+    st.pipe(wstream);
+
+    return d.promise;
+}
+
+// Find a filename available
+function getUniqueFilename(base, filename) {
+    if (!filename) {
+        filename = base;
+        base = "/";
+    }
+
+    filename = path.resolve(base, filename);
+    var ext = path.extname(filename);
+    filename = path.join(path.dirname(filename), path.basename(filename, ext));
+
+    var _filename = filename+ext;
+
+    var i = 0;
+    while (1) {
+        if (!fs.existsSync(filename)) break;
+        _filename = filename+"_"+i+ext;
+        i = i + 1;
+    }
+
+    return path.relative(base, _filename);
+}
+
+
+// List files in a directory
+function listFiles(root, options) {
+    options = _.defaults(options || {}, {
+        ignoreFiles: [],
+        ignoreRules: []
+    });
+
+    var d = Q.defer();
+
+    // Our list of files
+    var files = [];
+
+    var ig = Ignore({
+        path: root,
+        ignoreFiles: options.ignoreFiles
+    });
+
+    // Add extra rules to ignore common folders
+    ig.addIgnoreRules(options.ignoreRules, '__custom_stuff');
+
+    // Push each file to our list
+    ig.on('child', function (c) {
+        files.push(
+            c.path.substr(c.root.path.length + 1) + (c.props.Directory === true ? '/' : '')
+        );
+    });
+
+    ig.on('end', function() {
+        // Normalize paths on Windows
+        if(process.platform === 'win32') {
+            return d.resolve(files.map(function(file) {
+                return file.replace(/\\/g, '/');
+            }));
+        }
+
+        // Simply return paths otherwise
+        return d.resolve(files);
+    });
+
+    ig.on('error', d.reject);
+
+    return d.promise;
+}
+
+// Clean a folder without removing .git and .svn
+// Creates it if non existant
+function cleanFolder(root) {
+    if (!fs.existsSync(root)) return fsUtils.mkdirp(root);
+
+    return listFiles(root, {
+        ignoreFiles: [],
+        ignoreRules: [
+            // Skip Git and SVN stuff
+            '.git/',
+            '.svn/'
+        ]
+    })
+    .then(function(files) {
+        var d = Q.defer();
+
+        _.reduce(files, function(prev, file, i) {
+            return prev.then(function() {
+                var _file = path.join(root, file);
+
+                d.notify({
+                    i: i+1,
+                    count: files.length,
+                    file: _file
+                });
+                return fsUtils.remove(_file);
+            });
+        }, Q())
+        .then(function() {
+            d.resolve();
+        }, function(err) {
+            d.reject(err);
+        });
+
+        return d.promise;
+    });
+}
+
+module.exports = fsUtils;

lib/utils/git.js

@@ -0,0 +1,112 @@
+var  Q = require("q");
+var _ = require("lodash");
+var url = require("url");
+var tmp = require("tmp");
+var path = require("path");
+var crc = require("crc");
+var exec = Q.denodeify(require("child_process").exec);
+var URI = require("URIjs");
+
+var fs = require("./fs");
+
+var GIT_PREFIX = "git+";
+var GIT_TMP = null;
+
+
+// Check if an url is a git dependency url
+function checkGitUrl(giturl) {
+    return (giturl.indexOf(GIT_PREFIX) === 0);
+}
+
+// Validates a SHA in hexadecimal
+function validateSha(str) {
+    return (/[0-9a-f]{40}/).test(str);
+}
+
+// Parse and extract infos
+function parseGitUrl(giturl) {
+    var ref, uri, fileParts, filepath;
+
+    if (!checkGitUrl(giturl)) return null;
+    giturl = giturl.slice(GIT_PREFIX.length);
+
+    uri = new URI(giturl);
+    ref = uri.fragment() || "master";
+    uri.fragment(null);
+
+    // Extract file inside the repo (after the .git)
+    fileParts =uri.path().split(".git");
+    filepath = fileParts.length > 1? fileParts.slice(1).join(".git") : "";
+    if (filepath[0] == "/") filepath = filepath.slice(1);
+
+    // Recreate pathname without the real filename
+    uri.path(_.first(fileParts)+".git");
+
+    return {
+        host: uri.toString(),
+        ref: ref || "master",
+        filepath: filepath
+    };
+}
+
+// Clone a git repo from a specific ref
+function cloneGitRepo(host, ref) {
+    var isBranch = false;
+
+    ref = ref || "master";
+    if (!validateSha(ref)) isBranch = true;
+
+    return Q()
+
+    // Create temporary folder to store git repos
+    .then(function() {
+        if (GIT_TMP) return;
+        return fs.tmp.dir()
+            .then(function(_tmp) {
+                GIT_TMP = _tmp;
+            });
+    })
+
+    // Return or clone the git repo
+    .then(function() {
+        // Unique ID for repo/ref combinaison
+        var repoId = crc.crc32(host+"#"+ref).toString(16);
+
+        // Absolute path to the folder
+        var repoPath = path.resolve(GIT_TMP, repoId);
+
+        return fs.exists(repoPath)
+            .then(function(doExists) {
+                if (doExists) return;
+
+                // Clone repo
+                return exec("git clone "+host+" "+repoPath)
+                .then(function() {
+                    return exec("git checkout "+ref, { cwd: repoPath });
+                })
+            })
+            .thenResolve(repoPath);
+    });
+}
+
+
+// Get file from a git repo
+function resolveFileFromGit(giturl) {
+    if (_.isString(giturl)) giturl = parseGitUrl(giturl);
+    if (!giturl) return Q(null);
+
+    // Clone or get from cache
+    return cloneGitRepo(giturl.host, giturl.ref)
+    .then(function(repo) {
+
+        // Resolve relative path
+        return path.resolve(repo, giturl.filepath);
+    });
+};
+
+
+module.exports = {
+    checkUrl: checkGitUrl,
+    parseUrl: parseGitUrl,
+    resolveFile: resolveFileFromGit
+};

lib/utils/i18n.js

@@ -0,0 +1,72 @@
+var _ = require("lodash");
+var path = require("path");
+var fs = require("fs");
+
+var I18N_PATH = path.resolve(__dirname, "../../theme/i18n/")
+
+var getLocales = _.memoize(function() {
+    var locales = fs.readdirSync(I18N_PATH);
+    return _.chain(locales)
+        .map(function(local) {
+            local = path.basename(local, ".json");
+            return [local, _.extend({
+                direction: "ltr"
+            }, require(path.join(I18N_PATH, local)), {
+                id: local
+            })];
+        })
+        .object()
+        .value();
+});
+
+var getLanguages = function() {
+    return _.keys(getLocales());
+};
+
+var getByLanguage = function(lang) {
+    lang = normalizeLanguage(lang);
+    var locales = getLocales();
+    return locales[lang];
+};
+
+var compareLocales = function(lang, locale) {
+    var langMain = _.first(lang.split("-"));
+    var langSecond = _.last(lang.split("-"));
+
+    var localeMain = _.first(locale.split("-"));
+    var localeSecond = _.last(locale.split("-"));
+
+    if (locale == lang) return 100;
+    if (localeMain == langMain) return 50;
+    if (localeSecond == langSecond) return 20;
+    return 0;
+};
+
+var normalizeLanguage = _.memoize(function(lang) {
+    var locales = getLocales();
+    var language = _.chain(locales)
+        .values()
+        .map(function(locale) {
+            locale = locale.id;
+
+            return {
+                locale: locale,
+                score: compareLocales(lang, locale)
+            }
+        })
+        .filter(function(lang) {
+            return lang.score > 0;
+        })
+        .sortBy("score")
+        .pluck("locale")
+        .last()
+        .value();
+    return language || lang;
+});
+
+module.exports = {
+    getLocales: getLocales,
+    getLanguages: getLanguages,
+    getByLanguage: getByLanguage,
+    normalizeLanguage: normalizeLanguage
+};

lib/utils/images.js

@@ -0,0 +1,39 @@
+var _ = require("lodash");
+var Q = require("q");
+var fs = require("./fs");
+var spawn = require('spawn-cmd').spawn;
+
+var links = require("./links");
+
+// Convert a svg file
+var convertSVG = function(source, dest, options) {
+    if (!fs.existsSync(source)) return Q.reject(new Error("File doesn't exist: "+source));
+    var d = Q.defer();
+
+    options = _.defaults(options || {}, {
+
+    });
+
+    //var command = shellescape(['svgexport', source, dest]);
+    var child = spawn('svgexport', [source, dest]);
+
+    child.on("error", function(error) {
+        if (error.code == "ENOENT") error = new Error("Need to install 'svgexport' using 'npm install svgexport -g'");
+        return d.reject(error);
+    });
+
+    child.on("close", function(code) {
+        if (code == 0 && fs.existsSync(dest)) {
+            d.resolve();
+        } else {
+            d.reject(new Error("Error converting "+source+" into "+dest));
+        }
+    });
+
+    return d.promise;
+};
+
+module.exports = {
+    convertSVG: convertSVG,
+    INVALID: [".svg"]
+};

lib/utils/index.js

@@ -1,4 +0,0 @@
-module.exports = {
-    lang: require('./lang'),
-    links: require('./links')
-};

lib/utils/links.js

@@ -15,46 +15,68 @@ var isRelative = function(href) {
     try {
         var parsed = url.parse(href);
 
-        return !parsed.protocol && parsed.path && parsed.path[0] != '/';
+        return !!(!parsed.protocol && parsed.path);
     } catch(err) {}
 
     return true;
 };
 
+// Return true if the link is an achor
+var isAnchor = function(href) {
+    try {
+        var parsed = url.parse(href);
+        return !!(!parsed.protocol && !parsed.path && parsed.hash);
+    } catch(err) {}
+
+    return false;
+};
+
+// Normalize a path to be a link
+var normalizeLink = function(s) {
+    return s.replace(/\\/g, '/');
+};
+
 // Relative to absolute path
 // dir: directory parent of the file currently in rendering process
 // outdir: directory parent from the html output
 
 var toAbsolute = function(_href, dir, outdir) {
-    // Absolute file in source
-    _href = path.join(dir, _href);
+    if (isExternal(_href)) return _href;
 
-    // make it relative to output
-    _href = path.relative(outdir, _href);
+    // Path '_href' inside the base folder
+    var hrefInRoot = path.normalize(path.join(dir, _href));
+    if (_href[0] == "/") hrefInRoot = path.normalize(_href.slice(1));
 
-    if (process.platform === 'win32') {
-        _href = _href.replace(/\\/g, '/');
-    }
+    // Make it relative to output
+    _href = path.relative(outdir, hrefInRoot);
+
+    // Normalize windows paths
+    _href = normalizeLink(_href);
 
     return _href;
 };
 
 // Join links
-
 var join = function() {
     var _href = path.join.apply(path, arguments);
 
-    if (process.platform === 'win32') {
-        _href = _href.replace(/\\/g, '/');
-    }
-
-    return _href;
+    return normalizeLink(_href);
 };
 
+// Change extension
+var changeExtension = function(filename, newext) {
+    return path.join(
+        path.dirname(filename),
+        path.basename(filename, path.extname(filename))+newext
+    );
+};
 
 module.exports = {
+    isAnchor: isAnchor,
     isRelative: isRelative,
     isExternal: isExternal,
     toAbsolute: toAbsolute,
-    join: join
+    join: join,
+    changeExtension: changeExtension,
+    normalize: normalizeLink
 };

lib/utils/logger.js

@@ -0,0 +1,102 @@
+var _ = require('lodash');
+var util = require('util');
+var color = require('bash-color');
+
+var LEVELS = {
+    DEBUG: 0,
+    INFO: 1,
+    WARN: 2,
+    ERROR: 3,
+    DISABLED: 10
+};
+
+var COLORS = {
+    DEBUG: color.purple,
+    INFO: color.cyan,
+    WARN: color.yellow,
+    ERROR: color.red
+};
+
+module.exports = function(_write, logLevel) {
+    var logger = {};
+    var lastChar = '\n';
+    if (_.isString(logLevel)) logLevel = LEVELS[logLevel.toUpperCase()];
+
+    // Write a simple message
+    logger.write = function(msg) {
+        msg = msg.toString();
+        lastChar = _.last(msg);
+        return _write(msg);
+    };
+
+    // Format a message
+    logger.format = function() {
+        return util.format.apply(util, arguments);
+    };
+
+    // Write a line
+    logger.writeLn = function(msg) {
+        return this.write((msg || "")+"\n");
+    };
+
+    // Write a message with a certain level
+    logger.log = function(level) {
+        if (level < logLevel) return;
+
+        var levelKey = _.findKey(LEVELS, function(v) { return v == level; });
+        var args = Array.prototype.slice.apply(arguments, [1]);
+        var msg = logger.format.apply(logger, args);
+
+        if (lastChar == '\n') {
+            msg = COLORS[levelKey](levelKey.toLowerCase()+":")+" "+msg;
+        }
+
+        return logger.write(msg);
+    };
+    logger.logLn = function() {
+        if (lastChar != '\n') logger.write("\n");
+
+        var args = Array.prototype.slice.apply(arguments);
+        args.push("\n");
+        logger.log.apply(logger, args);
+    };
+
+    // Write a OK
+    logger.ok = function(level) {
+        var args = Array.prototype.slice.apply(arguments, [1]);
+        var msg = logger.format.apply(logger, args);
+        if (arguments.length > 1) {
+            logger.logLn(level, color.green('>> ') + msg.trim().replace(/\n/g, color.green('\n>> ')));
+        } else {
+            logger.log(level, color.green("OK"), "\n");
+        }
+    };
+
+    // Write an "FAIL"
+    logger.fail = function(level) {
+        return logger.log(level, color.red("ERROR")+"\n");
+    };
+
+    _.each(_.omit(LEVELS, 'DISABLED'), function(level, levelKey) {
+        levelKey = levelKey.toLowerCase();
+
+        logger[levelKey] = _.partial(logger.log, level);
+        logger[levelKey].ln = _.partial(logger.logLn, level);
+        logger[levelKey].ok = _.partial(logger.ok, level);
+        logger[levelKey].fail = _.partial(logger.fail, level);
+        logger[levelKey].promise = function(p) {
+            return p.
+            then(function(st) {
+                logger[levelKey].ok();
+                return st;
+            }, function(err) {
+                logger[levelKey].fail();
+                throw err;
+            });
+        }
+    });
+
+    return logger;
+};
+module.exports.LEVELS = LEVELS;
+module.exports.COLORS = COLORS;

lib/utils/navigation.js

@@ -27,27 +27,45 @@ function navigation(summary, files) {
     files = _.isArray(files) ? files : (_.isString(files) ? [files] : null);
 
     // List of all navNodes
-    // Flatten chapters, then add in default README node if ndeeded etc ...
+    // Flatten chapters, then add in default README node if needed etc ...
     var navNodes = flattenChapters(summary.chapters);
-    var prevNodes = [null].concat(navNodes.slice(0, -1));
-    var nextNodes = navNodes.slice(1).concat([null]);
 
     // Mapping of prev/next for a give path
-    var mapping = _.chain(_.zip(navNodes, prevNodes, nextNodes))
-    .map(function(nodes) {
-        var current = nodes[0], prev = nodes[1], next = nodes[2];
+    var mapping = _.chain(navNodes)
+    .map(function(current, i) {
+        var prev = null, next = null;
 
         // Skip if no path
-        if(!current.path) return null;
+        if(!current.exists) return null;
+
+        // Find prev
+        prev = _.chain(navNodes)
+            .slice(0, i)
+            .reverse()
+            .find(function(node) {
+                return node.exists && !node.external;
+            })
+            .value();
+
+        // Find next
+        next = _.chain(navNodes)
+            .slice(i+1)
+            .find(function(node) {
+                return node.exists && !node.external;
+            })
+            .value();
+
 
         return [current.path, {
+            index: i,
             title: current.title,
+            introduction: current.introduction,
             prev: prev,
             next: next,
             level: current.level,
         }];
     })
-    .filter()
+    .compact()
     .object()
     .value();
 

lib/utils/page.js

@@ -0,0 +1,372 @@
+var Q = require('q');
+var _ = require('lodash');
+var url = require('url');
+var path = require('path');
+var cheerio = require('cheerio');
+var domSerializer = require('dom-serializer');
+var request = require('request');
+var crc = require("crc");
+
+var links = require('./links');
+var imgUtils = require('./images');
+var fs = require('./fs');
+var batch = require('./batch');
+var code = require('./code');
+
+// Render a cheerio dom as html
+var renderDom = function($, dom, options) {
+    if (!dom && $._root && $._root.children) {
+        dom = $._root.children;
+    }
+
+    options = options|| dom.options || $._options;
+    return domSerializer(dom, options);
+};
+
+// Map of images that have been converted
+var imgConversionCache = {};
+
+function replaceText($, el, search, replace, text_only ) {
+    return $(el).each(function(){
+        var node = this.firstChild,
+            val,
+            new_val,
+
+            // Elements to be removed at the end.
+            remove = [];
+
+        // Only continue if firstChild exists.
+        if ( node ) {
+
+            // Loop over all childNodes.
+            do {
+                // Only process text nodes.
+                if ( node.nodeType === 3 ) {
+
+                    // The original node value.
+                    val = node.nodeValue;
+
+                    // The new value.
+                    new_val = val.replace( search, replace );
+
+                    // Only replace text if the new value is actually different!
+                    if ( new_val !== val ) {
+
+                        if ( !text_only && /</.test( new_val ) ) {
+                            // The new value contains HTML, set it in a slower but far more
+                            // robust way.
+                            $(node).before( new_val );
+
+                            // Don't remove the node yet, or the loop will lose its place.
+                            remove.push( node );
+                        } else {
+                            // The new value contains no HTML, so it can be set in this
+                            // very fast, simple way.
+                            node.nodeValue = new_val;
+                        }
+                    }
+                }
+
+            } while ( node = node.nextSibling );
+        }
+
+        // Time to remove those elements!
+        remove.length && $(remove).remove();
+    });
+};
+
+function pregQuote( str ) {
+    return (str+'').replace(/([\\\.\+\*\?\[\^\]\$\(\)\{\}\=\!\<\>\|\:])/g, "\\$1");
+};
+
+
+// Adapt an html snippet to be relative to a base folder
+function normalizeHtml(src, options) {
+    var $ = cheerio.load(src, {
+        // We should parse html without trying to normalize too much
+        xmlMode: false,
+
+        // SVG need some attributes to use uppercases
+        lowerCaseAttributeNames: false,
+        lowerCaseTags: false
+    });
+    var toConvert = [];
+    var svgContent = {};
+    var outputRoot = options.book.options.output;
+
+    imgConversionCache[outputRoot] = imgConversionCache[outputRoot] || {};
+
+    // Find svg images to extract and process
+    if (options.convertImages) {
+        $("svg").each(function() {
+            var content = renderDom($, $(this));
+            var svgId = _.uniqueId("svg");
+            var dest = svgId+".svg";
+
+            // Generate filename
+            dest = "/"+fs.getUniqueFilename(outputRoot, dest);
+
+            svgContent[dest] = '<?xml version="1.0" encoding="UTF-8"?>'+content;
+            $(this).replaceWith($("<img>").attr("src", dest));
+        });
+    }
+
+    // Find images to normalize
+    $("img").each(function() {
+        var origin = undefined;
+        var src = $(this).attr("src");
+        if (!src) return;
+        var isExternal = links.isExternal(src);
+
+        // Transform as relative to the bases
+        if (links.isRelative(src)) {
+            src = links.toAbsolute(src, options.base, options.output);
+        }
+
+        // Convert if needed
+        if (options.convertImages) {
+            // If image is external and ebook, then downlaod the images
+            if (isExternal) {
+                origin = src;
+                src = "/"+crc.crc32(origin).toString(16)+path.extname(origin);
+                src = links.toAbsolute(src, options.base, options.output);
+                isExternal = false;
+            }
+
+            var ext = path.extname(src);
+            var srcAbs = links.join("/", options.base, src);
+
+            // Test image extension
+            if (_.contains(imgUtils.INVALID, ext)) {
+                if (imgConversionCache[outputRoot][srcAbs]) {
+                    // Already converted
+                    src = imgConversionCache[outputRoot][srcAbs];
+                } else {
+                    // Not converted yet
+                    var dest = "";
+
+                    // Replace extension
+                    dest = links.join(path.dirname(srcAbs), path.basename(srcAbs, ext)+".png");
+                    dest = dest[0] == "/"? dest.slice(1) : dest;
+
+                    // Get a name that doesn't exists
+                    dest = fs.getUniqueFilename(outputRoot, dest);
+
+                    options.book.log.debug.ln("detect invalid image (will be converted to png):", srcAbs);
+
+                    // Add to cache
+                    imgConversionCache[outputRoot][srcAbs] = "/"+dest;
+
+                    // Push to convert
+                    toConvert.push({
+                        origin: origin,
+                        content: svgContent[srcAbs],
+                        source: isExternal? srcAbs : path.join("./", srcAbs),
+                        dest: path.join("./", dest)
+                    });
+
+                    src = links.join("/", dest);
+                }
+
+                // Reset as relative to output
+                src = links.toAbsolute(src, options.base, options.output);
+            }
+
+            else if (origin) {
+                // Need to downlaod image
+                toConvert.push({
+                    origin: origin,
+                    source: path.join("./", srcAbs)
+                });
+            }
+        }
+
+        $(this).attr("src", src);
+    });
+
+    // Normalize links
+    $("a").each(function() {
+        var href = $(this).attr("href");
+        if (!href) return;
+
+        if (links.isAnchor(href)) {
+            // Keep it as it is
+        } else if (links.isRelative(href)) {
+            var parts = url.parse(href);
+            var absolutePath = path.join(options.base, parts.pathname);
+            var anchor = parts.hash || "";
+
+            // If is in navigation relative: transform as content
+            if (options.navigation[absolutePath]) {
+                absolutePath = options.book.contentLink(absolutePath);
+            }
+
+            // Transform as absolute
+            href = links.toAbsolute("/"+absolutePath, options.base, options.output)+anchor;
+        } else {
+            // External links
+            $(this).attr("target", "_blank");
+        }
+
+        // Transform extension
+        $(this).attr("href", href);
+    });
+
+    // Highlight code blocks
+    $("code").each(function() {
+        // Extract language
+        var lang = _.chain(
+                ($(this).attr("class") || "").split(" ")
+            )
+            .map(function(cl) {
+                if (cl.search("lang-") === 0) return cl.slice("lang-".length);
+                return null;
+            })
+            .compact()
+            .first()
+            .value();
+
+        if (lang) {
+            var html = code.highlight(
+                lang,
+                $(this).text()
+            );
+
+            $(this).html(html);
+        }
+    });
+
+    // Replace glossary terms
+    _.each(options.glossary, function(term) {
+        var r =  new RegExp( "\\b(" + pregQuote(term.name.toLowerCase()) + ")\\b" , 'gi' );
+        var includedInFiles = false;
+
+        $("*").each(function() {
+            // Ignore codeblocks
+            if (_.contains(["code", "pre"], this.name.toLowerCase())) return;
+
+            replaceText($, this, r, function(match) {
+                // Add to files index in glossary
+                if (!includedInFiles) {
+                    includedInFiles = true;
+                    term.files = term.files || [];
+                    term.files.push(options.navigation[options.input]);
+                }
+                return "<a href='"+links.toAbsolute("/GLOSSARY.html", options.base, options.output)+"#"+term.id+"' class='glossary-term' title='"+_.escape(term.description)+"'>"+match+"</a>";
+            });
+        });
+    });
+
+    return {
+        html: renderDom($),
+        images: toConvert
+    };
+};
+
+// Convert svg images to png
+function convertImages(images, options) {
+    if (!options.convertImages) return Q();
+
+    var downloaded = [];
+    options.book.log.debug.ln("convert ", images.length, "images to png");
+
+    return batch.execEach(images, {
+        max: 100,
+        fn: function(image) {
+            var imgin = path.resolve(options.book.options.output, image.source);
+
+            return Q()
+
+            // Write image if need to be download
+            .then(function() {
+                if (!image.origin && !_.contains(downloaded, image.origin)) return;
+                options.book.log.debug("download image", image.origin, "...");
+                downloaded.push(image.origin);
+                return options.book.log.debug.promise(fs.writeStream(imgin, request(image.origin)));
+            })
+
+            // Write svg if content
+            .then(function() {
+                if (!image.content) return;
+                return fs.writeFile(imgin, image.content);
+            })
+
+            // Convert
+            .then(function() {
+                if (!image.dest) return;
+                var imgout = path.resolve(options.book.options.output, image.dest);
+                options.book.log.debug("convert image", image.source, "to", image.dest, "...");
+                return options.book.log.debug.promise(imgUtils.convertSVG(imgin, imgout));
+            });
+        }
+    })
+    .then(function() {
+        options.book.log.debug.ok(images.length+" images converted with success");
+    });
+};
+
+
+// Adapt page content to be relative to a base folder
+function normalizePage(sections, options) {
+    options = _.defaults(options || {}, {
+        // Current book
+        book: null,
+
+        // Do we need to convert svg?
+        convertImages: false,
+
+        // Current file path
+        input: ".",
+
+        // Navigation to use to transform path
+        navigation: {},
+
+        // Directory parent of the file currently in rendering process
+        base: "./",
+
+        // Directory parent from the html output
+        output: "./",
+
+        // Glossary terms
+        glossary: []
+    });
+
+    // List of images to convert
+    var toConvert = [];
+
+    sections = _.map(sections, function(section) {
+        if (section.type != "normal") return section;
+
+        var out = normalizeHtml(section.content, options);;
+
+        toConvert = toConvert.concat(out.images);
+        section.content = out.html;
+        return section;
+    });
+
+    return Q()
+    .then(function() {
+        toConvert = _.uniq(toConvert, 'source');
+        return convertImages(toConvert, options);
+    })
+    .thenResolve(sections);
+};
+
+// Extract text from sections
+function extractText(sections) {
+    return _.reduce(sections, function(prev, section) {
+        if (section.type != "normal") return prev;
+
+        var $ = cheerio.load(section.content);
+        $("*").each(function() {
+            prev = prev+" "+$(this).text();
+        });
+
+        return prev;
+    }, "");
+};
+
+module.exports = {
+    normalize: normalizePage,
+    extractText: extractText
+};

lib/utils/progress.js

@@ -7,10 +7,18 @@ var calculProgress = function(navigation, current) {
     var done = true;
 
     var chapters = _.chain(navigation)
+
+    // Transform as array
     .map(function(nav, path) {
         nav.path = path;
         return nav;
     })
+
+    // Sort entries
+    .sortBy(function(nav) {
+        return nav.index;
+    })
+
     .map(function(nav, i) {
         // Calcul percent
         nav.percent = (i * 100) / Math.max((n - 1), 1);
@@ -44,4 +52,4 @@ var calculProgress = function(navigation, current) {
     };
 }
 
-module.exports = calculProgress;
+module.exports = calculProgress;

lib/utils/string.js

@@ -20,7 +20,13 @@ function optionsToShellArgs(options) {
     .join(" ");
 }
 
+function escapeRegex(str) {
+    return str.replace(/[\-\[\]\/\{\}\(\)\*\+\?\.\\\^\$\|]/g, "\\$&");
+}
+
 module.exports = {
+    escapeRegex: escapeRegex,
     escapeShellArg: escapeShellArg,
-    optionsToShellArgs: optionsToShellArgs
+    optionsToShellArgs: optionsToShellArgs,
+    toLowerCase: String.prototype.toLowerCase.call.bind(String.prototype.toLowerCase)
 };

lib/utils/watch.js

@@ -0,0 +1,38 @@
+var Q = require('q');
+var _ = require('lodash');
+var path = require('path');
+var Gaze = require('gaze').Gaze;
+
+var parsers = require('gitbook-parsers')
+
+function watch(dir) {
+    var d = Q.defer();
+    dir = path.resolve(dir);
+
+    var toWatch = [
+        "book.json", "book.js"
+    ];
+
+    _.each(parsers.extensions, function(ext) {
+        toWatch.push("**/*"+ext);
+    });
+
+    var gaze = new Gaze(toWatch, {
+        cwd: dir
+    });
+
+    gaze.once("all", function(e, filepath) {
+        gaze.close();
+
+        d.resolve(filepath);
+    });
+    gaze.once("error", function(err) {
+        gaze.close();
+
+        d.reject(err);
+    });
+
+    return d.promise;
+}
+
+module.exports = watch;

package.json

@@ -1,61 +1,65 @@
 {
     "name": "gitbook",
-    "version": "1.5.0",
-    "homepage": "http://www.gitbook.io/",
+    "version": "2.0.0",
+    "homepage": "https://www.gitbook.com",
     "description": "Library and cmd utility to generate GitBooks",
     "main": "lib/index.js",
     "dependencies": {
         "q": "1.0.1",
-        "lodash": "2.4.1",
-        "kramed": "0.4.3",
-        "kramed-text-renderer": "0.2.1",
-        "lunr": "0.5.2",
-        "swig": "1.3.2",
-        "send": "0.2.0",
-        "fstream-ignore": "0.0.7",
-        "commander": "2.2.0",
-        "graceful-fs": "3.0.2",
-        "fs-extra": "0.10.0",
-        "highlight.js": "8.3.0",
-        "tmp": "0.0.23",
+        "lunr": "0.5.7",
+        "lodash": "3.5.0",
+        "graceful-fs": "3.0.5",
+        "resolve": "0.6.3",
+        "fs-extra": "0.16.5",
+        "fstream-ignore": "1.0.2",
+        "gitbook-parsers": "0.7.2",
+        "nunjucks": "git+https://github.com/mozilla/nunjucks.git#0f8b21b8df7e8e852b2e1889388653b7075f0d09",
+        "nunjucks-autoescape": "0.1.1",
+        "nunjucks-filter": "0.1.0",
+        "nunjucks-i18n": "1.0.0",
         "semver": "2.2.1",
         "npmi": "0.1.1",
-        "gaze": "~0.5.1",
-        "resolve": "0.6.3",
-        "tiny-lr-fork": "0.0.5",
-        "gitbook-plugin": "0.0.2",
-        "gitbook-plugin-mathjax": "0.0.6",
+        "cheerio": "0.19.0",
         "gitbook-plugin-livereload": "0.0.1",
-        "gitbook-plugin-exercises": "1.0.0",
-        "gitbook-plugin-quizzes": "1.0.0"
+        "gaze": "~0.5.1",
+        "send": "0.2.0",
+        "tiny-lr": "0.1.5",
+        "tmp": "0.0.24",
+        "crc": "3.2.1",
+        "bash-color": "0.0.3",
+        "URIjs": "1.14.1",
+        "request": "2.51.0",
+        "npm": "2.4.1",
+        "dom-serializer": "0.1.0",
+        "spawn-cmd": "0.0.2",
+        "highlight.js": "8.4.0"
     },
     "devDependencies": {
-        "mocha": "1.18.2",
+        "mocha": "2.2.1",
+        "should": "5.2.0",
         "grunt": "~0.4.2",
         "grunt-cli": "0.1.11",
         "grunt-contrib-copy": "0.5.0",
         "grunt-contrib-less": "~0.5.0",
         "grunt-contrib-requirejs": "0.4.1",
-        "grunt-bower-install-simple": "0.9.2",
-        "grunt-browserify": "3.1.0",
-        "grunt-contrib-uglify": "0.6.0"
+        "grunt-bower-install-simple": "0.9.2"
     },
     "scripts": {
-        "test": "export TESTING=true; mocha --reporter list"
-    },
-    "bin": {
-        "gitbook": "./bin/gitbook.js"
+        "test": "node_modules/.bin/mocha --reporter spec --timeout 15000"
     },
     "repository": {
         "type": "git",
         "url": "https://github.com/GitbookIO/gitbook.git"
     },
+    "bin": {
+        "gitbook": "./bin/gitbook.js"
+    },
     "keywords": [
         "git",
         "book",
         "gitbook"
     ],
-    "author": "FriendCode <contact@friendco.de>",
+    "author": "FriendCode Inc. <contact@gitbook.com>",
     "license": "Apache 2",
     "bugs": {
         "url": "https://github.com/GitbookIO/gitbook/issues"
@@ -63,11 +67,11 @@
     "contributors": [
         {
             "name": "Aaron O'Mullan",
-            "email": "aaron.omullan@gmail.com"
+            "email": "aaron@gitbook.com"
         },
         {
             "name": "Samy Pessé",
-            "email": "samypesse@gmail.com"
+            "email": "samy@gitbook.com"
         }
     ]
 }

test/assertions.js

@@ -0,0 +1,52 @@
+var _ = require('lodash');
+var fs = require('fs');
+var path = require('path');
+var should = require('should');
+var cheerio = require('cheerio');
+
+should.Assertion.add('file', function(file, description) {
+    this.params = { actual: this.obj.toString(), operator: 'have file ' + file, message: description };
+
+    this.obj.should.have.property('options').which.is.an.Object;
+    this.obj.options.should.have.property('output').which.is.a.String;
+    this.assert(fs.existsSync(path.resolve(this.obj.options.output, file)));
+});
+
+should.Assertion.add('jsonfile', function(file, description) {
+    this.params = { actual: this.obj.toString(), operator: 'have valid jsonfile ' + file, message: description };
+
+    this.obj.should.have.property('options').which.is.an.Object;
+    this.obj.options.should.have.property('output').which.is.a.String;
+    this.assert(JSON.parse(fs.readFileSync(path.resolve(this.obj.options.output, file), { encoding: "utf-8" })));
+});
+
+should.Assertion.add('html', function(rules, description) {
+    this.params = { actual: "HTML string", operator: 'valid html', message: description };
+    var $ = cheerio.load(this.obj);
+
+    _.each(rules, function(validations, query) {
+        validations = _.defaults(validations || {}, {
+            count: 1,
+            attributes: {},
+            trim: false,
+            text: undefined
+        });
+
+        var $el = $(query);
+
+        // Test number of elements
+        $el.should.have.lengthOf(validations.count);
+
+        // Test text
+        if (validations.text !== undefined) {
+            var text = $el.text();
+            if (validations.trim) text = text.trim();
+            text.should.be.equal(validations.text);
+        }
+
+        // Test attributes
+        _.each(validations.attributes, function(value, name) {
+            $el.attr(name).should.be.equal(value);
+        });
+    });
+});

test/books/basic/README.md

@@ -0,0 +1,3 @@
+# Readme
+
+Default description for the book.

test/books/basic/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/config-js/README.md

@@ -0,0 +1 @@
+# Readme

test/books/config-js/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/config-js/book.js

@@ -0,0 +1,3 @@
+module.exports = {
+    "title": "js-config"
+};

test/books/config-json/README.md

@@ -0,0 +1 @@
+# Readme

test/books/config-json/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/config-json/book.json

@@ -0,0 +1,3 @@
+{
+    "title": "json-config"
+}

test/books/conrefs/README.md

@@ -0,0 +1,10 @@
+# Readme
+
+### Relative
+
+<p id="t1">{% include "./hello.md" %}</p>
+
+### Git
+
+<p id="t2">{% include "git+https://gist.github.com/69ea4542e4c8967d2fa7.git/test.md" %}</p>
+<p id="t3">{% include "git+https://gist.github.com/69ea4542e4c8967d2fa7.git/test2.md" %}</p>

test/books/conrefs/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/conrefs/hello.md

@@ -0,0 +1 @@
+Hello World

test/books/glossary/GLOSSARY.md

@@ -0,0 +1,9 @@
+# Glossary
+
+## test
+
+Just a simple and easy to understand test.
+
+## hakunamatata
+
+This word is not present in the content.

test/books/glossary/README.md

@@ -0,0 +1,8 @@
+# Readme
+
+The word test should be replaced by a glossary link.
+
+```
+But the word test should not be replaced in code blocks
+```
+

test/books/glossary/SUMMARY.md

@@ -0,0 +1,3 @@
+# Summary
+
+* [Page](folder/PAGE.md)

test/books/glossary/folder/PAGE.md

@@ -0,0 +1,3 @@
+# Page
+
+Just a page in a sub-directory to test relative link to glossary.

test/books/images/README.md

@@ -0,0 +1,18 @@
+# Tests for Images
+
+# SVG relative image
+
+
+### Convert SVG to PNG
+![test.svg](test.svg)
+
+### Should only convert it once
+![test.svg](test.svg)
+
+### Download remote images
+![remote image](http://upload.wikimedia.org/wikipedia/commons/b/b0/NewTux.svg)
+
+### Remote images with same filename
+![youtube1](http://img.youtube.com/vi/9bZkp7q19f0/0.jpg)
+![youtube2](http://img.youtube.com/vi/IkV2HQLAKHY/0.jpg)
+

test/books/images/SUMMARY.md

@@ -0,0 +1,3 @@
+# Summary
+
+* [Page](./folder/PAGE.md)

test/books/images/folder/PAGE.md

@@ -0,0 +1,3 @@
+### Images from other folder
+![test.svg](../test.svg)
+

test/books/images/test.svg

@@ -0,0 +1,11 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1 Tiny//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11-tiny.dtd">
+<svg version="1.1" baseProfile="tiny" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" id="svg-root" width="100%" height="100%" viewBox="0 0 480 360">
+    <title id="test-title">basic SVG tiny doc</title>
+    <g id="test-body-content">
+        <text font-family="Arial" font-size="14" text-anchor="middle" x="225" y="25">hello world</text>
+    </g>
+    <text id="revision" x="10" y="340" font-size="40" stroke="none" fill="black">Revision: 1.1</text>
+    <rect id="test-frame" x="1" y="1" width="478" height="358" fill="none" stroke="#000000"/>
+</svg>
+

test/books/languages/LANGS.md

@@ -0,0 +1,4 @@
+# Language
+
+* [English](en)
+* [French](fr)

test/books/languages/README.md

@@ -0,0 +1 @@
+# Readme

test/books/languages/en/README.md

@@ -0,0 +1 @@
+Hello

test/books/languages/en/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/languages/fr/README.md

@@ -0,0 +1 @@
+Bonjour

test/books/languages/fr/SUMMARY.md

@@ -0,0 +1 @@
+# Summary

test/books/summary/PAGE1.md

@@ -0,0 +1 @@
+# Page 1

test/books/summary/README.md

@@ -0,0 +1 @@
+# Readme

test/books/summary/SUMMARY.md

@@ -0,0 +1,5 @@
+# Summary
+
+* [Page 1](./PAGE1.md)
+* [Page 2](./folder/PAGE2.md)
+* [Don't exists](./NOTFOUND.md)

test/books/summary/folder/PAGE2.md

@@ -0,0 +1 @@
+# Page 2

test/configuration.js

@@ -0,0 +1,32 @@
+var fs = require('fs');
+var path = require('path');
+
+describe('Configuration', function () {
+    it('should extract default title from README', function() {
+        return books.parse("basic")
+            .then(function(book) {
+                book.options.title.should.be.equal("Readme");
+            });
+    });
+
+    it('should extract default description from README', function() {
+        return books.parse("basic")
+            .then(function(book) {
+                book.options.description.should.be.equal("Default description for the book.");
+            });
+    });
+
+    it('should correctly load from json (book.json)', function() {
+        return books.parse("config-json")
+            .then(function(book) {
+                book.options.title.should.be.equal("json-config");
+            });
+    });
+
+    it('should correctly load from JavaScript (book.js)', function() {
+        return books.parse("config-js")
+            .then(function(book) {
+                book.options.title.should.be.equal("js-config");
+            });
+    });
+});

test/conrefs.js

@@ -0,0 +1,43 @@
+var fs = require('fs');
+var path = require('path');
+
+describe('ConRefs', function () {
+    var book, readme;
+
+    before(function() {
+        return books.generate("conrefs", "website")
+            .then(function(_book) {
+                book = _book;
+
+                readme = fs.readFileSync(
+                    path.join(book.options.output, "index.html"),
+                    { encoding: "utf-8" }
+                );
+            });
+    });
+
+    it('should handle local references', function() {
+        readme.should.be.html({
+            ".page-inner p#t1": {
+                count: 1,
+                text: "Hello World",
+                trim: true
+            }
+        });
+    });
+
+    it('should handle git references', function() {
+        readme.should.be.html({
+            ".page-inner p#t2": {
+                count: 1,
+                text: "Hello from git",
+                trim: true
+            },
+            ".page-inner p#t3": {
+                count: 1,
+                text: "First Hello. Hello from git",
+                trim: true
+            }
+        });
+    });
+});

test/ebook.js

@@ -0,0 +1,24 @@
+var fs = require('fs');
+var path = require('path');
+
+describe('eBook generator', function () {
+    describe('Basic Book', function() {
+        var book;
+
+        before(function() {
+            return books.generate("basic", "ebook")
+                .then(function(_book) {
+                    book = _book;
+                });
+        });
+
+        it('should correctly output a SUMMARY.html', function() {
+            book.should.have.file("SUMMARY.html");
+        });
+
+        it('should correctly copy assets', function() {
+            book.should.have.file("gitbook");
+            book.should.have.file("gitbook/style.css");
+        });
+    });
+});

test/fixtures/ALTERNATIVE_SUMMARY.md

@@ -1,13 +0,0 @@
-# Summary
-
-* [Custom name for Introduction](README.md)
-* [Chapter 1](chapter-1/README.md)
-    * [Article 1](chapter-1/ARTICLE1.md)
-    * [Article 2](chapter-1/ARTICLE2.md)
-        * [article 1.2.1](chapter-1/ARTICLE-1-2-1.md)
-        * [article 1.2.2](chapter-1/ARTICLE-1-2-2.md)
-* [Chapter 2](chapter-2/README.md)
-* [Chapter 3](chapter-3/README.md)
-* [Chapter 4](chapter-4/README.md)
-    * Unfinished article
-* Unfinished Chapter

test/fixtures/FALSE_QUIZ.md

@@ -1,120 +0,0 @@
-# Learn Node.js by Example
-
-
-## Requirements
-
-- [x] A computer with internet access
-- [ ] Time: 30h (e.g. 2 months 30 mins per day or **1 week intensive**)
-
-
-## What is Node.js ?
-
-Node.js lets you *easily* build networked software (websites, applications "apps",
-using JavaScript).
-
-Its not "*point-and-click*" like WordPress, SquareSpace or Salesforce;
-you will need to write some "code". But as I will demonstrate, that's
-a *lot* easier than it sounds and gives you more power/flexibility
-and puts you in full control.
-
-## Node.js is not "Version 1.0" yet can I used it in Production?
-
-Yes! Some of the biggest organisations/companies in the world
-are using Node.js in Production systems:
-
-[Alibaba](https://github.com/alibaba/node-hbase-client),
-[Ajax.org](Ajax.org),
-[Box.com](http://tech.blog.box.com/2014/06/node-js-high-availability-at-box/), British Sky Broadcasting (Sky/Now TV),
-CNN,
-[Cloudup](https://cloudup.com/),
-Conde Nast,
-[DirectTV](http://strongloop.com/strongblog/node-summit-media-companies-embrace-node-js-for-rapidly-developing-responsive-apps/),
-[Dow Jones](http://nodejs.org/industry),
-eBay,
-[FeedHenry](http://www.feedhenry.com/),
-[GitHub](https://twitter.com/github/status/16979699217465344),
-[Google](http://venturebeat.com/2012/01/24/node-at-google-mozilla-yahoo/),
-[Groupon](http://nodeup.com/fiftyeight),
-HBO,
-Help.com,
-[HP](https://github.com/joyent/node/wiki/Projects,-Applications,-and-Companies-Using-Node),
-iTV,
-[Joyent](https://www.joyent.com/) (duh!),
-[Klout](https://klout.com),
-LinkedIn,
-McDonalds,
-[Medium](https://medium.com/the-story),
-Mozilla,
-NetFlix,
-[OpenTable](http://hapijs.com/community),
-PayPal,
-Pearson,
-~~Q~~,
-[Revolt](http://revolt.tv/),
-[Square](https://modulus.io/companies-using-node),
-Tesco,
-ThomasCook,
-Trello,
-Uber,
-Voxer,
-Walmart,
-Wikimedia (in progress of moving to SOA with node!)
-Yahoo,
-Yammer,
-[Yandex](https://www.youtube.com/watch?v=zdCxgdH4wZo),
-[Zendesk](http://radar.zendesk.com/)
-
-Want more? See: http://nodejs.org/industry/ and <br />
-https://github.com/joyent/node/wiki/Projects,-Applications,-and-Companies-Using-Node
-
-
-# Try it!
-
-## Download & Install
-
-> http://nodejs.org/download/
-
-
-## Node.js (Core) API
-
-The node.js ("core") has many useful modules.
-
-Bookmark: [http://nodejs.org/api](http://nodejs.org/api/) (you will come back to it)
-
-
-
-## Stability (Can we use it?)
-
-> *Which node.js* ***core*** *package(s) can/should I use?*
-
-Every core module has a
-["***Stability Index***"](http://nodejs.org/api/documentation.html#documentation_stability_index)
-rating on the node.js API.
-
-**General rule**: If you are being *paid* to write code
-that runs in node.js, <br /> pick core modules/methods
-with stability **Stable**, **API Frozen** and **Locked**.
-
-![Node.js Stability Index](http://i.imgur.com/xIroFrS.png)
-
-
-### Examples
-
-- [**cluster**](http://nodejs.org/api/cluster.html) is ***Experimental*** - don't use
-- [**domain**](http://nodejs.org/api/domain.html) is ***Unstable*** - don't use
-- [**path**](http://nodejs.org/api/path.html) is ***Stable*** - use
-- [**events**](http://nodejs.org/api/events.html) is ***Frozen*** - use
-- [**assert**](http://nodejs.org/api/assert.html) is ***Locked*** - use
-
-Core Modules to Learn
-
-- path
-- os
-
-
-
-Community Modules to Learn:
-
-- [jscs](https://www.npmjs.org/package/jscs) - code style checker
-- [q](https://www.npmjs.org/package/q) - promises library
-- [nd](https://www.npmjs.org/package/nd) - view documentation for a module

test/fixtures/GITHUB_LINKS.md

@@ -1,3 +0,0 @@
-# Nice course
-
-Check out this source file [in C++](../src/something.cpp)