From 22699a979dad2e61a1d5a923c473c56d50f3c8dc Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 14 Jul 2013 16:12:06 -0700 Subject: Updated authors in README. --- README | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index 43e522976..cade75cee 100644 --- a/README +++ b/README @@ -2738,7 +2738,8 @@ Puneeth Chaganti, Paul Rivier, rodja.trappe, Bradley Kuhn, thsutton, Nathan Gass, Jonathan Daugherty, Jérémy Bobbio, Justin Bogner, qerub, Christopher Sawicki, Kelsey Hightower, Masayoshi Takahashi, Antoine Latter, Ralf Stephan, Eric Seidel, B. Scott Michel, Gavin Beatty, -Sergey Astanin, Arlo O'Keeffe, Denis Laxalde, Brent Yorgey. +Sergey Astanin, Arlo O'Keeffe, Denis Laxalde, Brent Yorgey, David Lazar, +Jamie F. Olson. [markdown]: http://daringfireball.net/projects/markdown/ [reStructuredText]: http://docutils.sourceforge.net/docs/ref/rst/introduction.html -- cgit v1.2.3 From 595149a9bcb12aa832065f591058ab963c9f237e Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Mon, 15 Jul 2013 19:32:08 -0700 Subject: Fixed YAML code block in README. --- README | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) (limited to 'README') diff --git a/README b/README index cade75cee..adec7c0e7 100644 --- a/README +++ b/README @@ -1794,19 +1794,19 @@ if a title contains a colon, it must be quoted. The pipe character literally, without need for escaping. This form is necessary when the field contains blank lines: - --- - title: 'This is the title: it contains a colon' - author: - - name: Author One - affiliation: University of Somewhere - - name: Author Two - affiliation: University of Nowhere - tags: [nothing, nothingness] - abstract: | - This is the abstract. - - It consists of two paragraphs. - ... + --- + title: 'This is the title: it contains a colon' + author: + - name: Author One + affiliation: University of Somewhere + - name: Author Two + affiliation: University of Nowhere + tags: [nothing, nothingness] + abstract: | + This is the abstract. + + It consists of two paragraphs. + ... Template variables will be set from the metadata. Thus, for example, in writing HTML, the variable `abstract` will be set to the HTML -- cgit v1.2.3 From 7d75b913bd45c41abefbc055163cf455c4b37b65 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Wed, 17 Jul 2013 15:06:43 -0700 Subject: LaTeX/Beamer template: Added `classoption` variable. This is intended for class options like `oneside`; it may be repeated with different options. Thanks to Oliver Matthews. --- README | 3 +++ data/templates | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index adec7c0e7..f86de8cd7 100644 --- a/README +++ b/README @@ -768,6 +768,9 @@ as `title`, `author`, and `date`) as well as the following: : font size (10pt, 11pt, 12pt) for LaTeX documents `documentclass` : document class for LaTeX documents +`classoption` +: option for LaTeX documentclass, e.g. `oneside`; may be repeated + for multiple options `geometry` : options for LaTeX `geometry` class, e.g. `margin=1in`; may be repeated for multiple options diff --git a/data/templates b/data/templates index cb23306c2..24dab8e06 160000 --- a/data/templates +++ b/data/templates @@ -1 +1 @@ -Subproject commit cb23306c2721d9c1f918f057d7402e03e079476b +Subproject commit 24dab8e06ec3cdc66a6b6db0ebe17a586c2c67f6 -- cgit v1.2.3 From 6c2e76ac617e5972db5d118525e7f6f59f43caac Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Wed, 17 Jul 2013 15:38:56 -0700 Subject: Added `ignore_line_breaks` markdown extension. This causes intra-paragraph line breaks to be ignored, rather than being treated as hard line breaks or spaces. This is useful for some East Asian languages, where spaces aren't used between words, but text is separated into lines for readability. --- README | 6 ++++++ src/Text/Pandoc/Options.hs | 1 + src/Text/Pandoc/Readers/Markdown.hs | 1 + 3 files changed, 8 insertions(+) (limited to 'README') diff --git a/README b/README index f86de8cd7..a65e22a70 100644 --- a/README +++ b/README @@ -2416,6 +2416,12 @@ example, `markdown+hard_line_breaks` is markdown with hard line breaks. Causes all newlines within a paragraph to be interpreted as hard line breaks instead of spaces. +**Extension: `ignore_line_breaks`**\ +Causes newlines within a paragraph to be ignored, rather than being +treated as spaces or as hard line breaks. This option is intended for +use with East Asian languages where spaces are not used between words, +but text is divided into lines for readability. + **Extension: `tex_math_single_backslash`**\ Causes anything between `\(` and `\)` to be interpreted as inline TeX math, and anything between `\[` and `\]` to be interpreted diff --git a/src/Text/Pandoc/Options.hs b/src/Text/Pandoc/Options.hs index c9a5e27da..61a85cf6e 100644 --- a/src/Text/Pandoc/Options.hs +++ b/src/Text/Pandoc/Options.hs @@ -92,6 +92,7 @@ data Extension = | Ext_superscript -- ^ Superscript using ^this^ syntax | Ext_subscript -- ^ Subscript using ~this~ syntax | Ext_hard_line_breaks -- ^ All newlines become hard line breaks + | Ext_ignore_line_breaks -- ^ Newlines in paragraphs are ignored | Ext_literate_haskell -- ^ Enable literate Haskell conventions | Ext_abbreviations -- ^ PHP markdown extra abbreviation definitions | Ext_auto_identifiers -- ^ Automatic identifiers for headers diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs index a3500fbcf..1aa392162 100644 --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -1566,6 +1566,7 @@ endline = try $ do notFollowedBy' bulletListStart notFollowedBy' anyOrderedListStart (guardEnabled Ext_hard_line_breaks >> return (return B.linebreak)) + <|> (guardEnabled Ext_ignore_line_breaks >> return mempty) <|> (return $ return B.space) -- -- cgit v1.2.3 From df3546068e8a04b83c7cc3aad31b286c37b86d18 Mon Sep 17 00:00:00 2001 From: David Röthlisberger Date: Sat, 20 Jul 2013 13:02:35 +0100 Subject: README: Fix minor typo --- README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index a65e22a70..539f40a0f 100644 --- a/README +++ b/README @@ -892,7 +892,7 @@ There are two kinds of headers, Setext and atx. ### Setext-style headers ### A setext-style header is a line of text "underlined" with a row of `=` signs -(for a level one header) of `-` signs (for a level two header): +(for a level one header) or `-` signs (for a level two header): A level-one header ================== -- cgit v1.2.3 From a5df042e28e01dc6e70bbd1019b15d94cf9f06a9 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sat, 20 Jul 2013 10:52:00 -0700 Subject: LaTeX template: Added `biblio-style` variable. Closes #920. --- README | 2 ++ data/templates | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index 539f40a0f..d9b003344 100644 --- a/README +++ b/README @@ -790,6 +790,8 @@ as `title`, `author`, and `date`) as well as the following: : color for citation links in LaTeX documents `links-as-notes` : causes links to be printed as footnotes in LaTeX documents +`biblio-style` +: bibliography style in LaTeX, when used with `--natbib` `section` : section number in man pages `header` diff --git a/data/templates b/data/templates index 24dab8e06..120bed772 160000 --- a/data/templates +++ b/data/templates @@ -1 +1 @@ -Subproject commit 24dab8e06ec3cdc66a6b6db0ebe17a586c2c67f6 +Subproject commit 120bed772b4a235a0d17ea4560a682e23ac293ac -- cgit v1.2.3 From 85dacbb282f34dc45a810d7c7b5a2d639b70e1f5 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Tue, 23 Jul 2013 22:52:15 -0700 Subject: Added `--filter` option. This makes it easier to use JSON filters. Instead of doing pandoc -t json | ./filter | pandoc -f json you can just do pandoc --filter ./filter --- README | 8 ++++++++ pandoc.cabal | 1 + pandoc.hs | 25 +++++++++++++++++++++---- 3 files changed, 30 insertions(+), 4 deletions(-) (limited to 'README') diff --git a/README b/README index d9b003344..83873a92e 100644 --- a/README +++ b/README @@ -259,6 +259,14 @@ Reader options require different kinds of images. Currently this option only affects the markdown and LaTeX readers. +`--filter=`*PATH* +: Specify an executable to be used as a filter transforming the + Pandoc AST after the input is parsed and before the output is + written. The executable should read JSON from stdin and write + JSON to stdout. The JSON must be formatted like pandoc's own + JSON input and output. Filters may be most easily created in Haskell, + using the utility function `toJsonFilter` from `Text.Pandoc`. + `--normalize` : Normalize the document after reading: merge adjacent `Str` or `Emph` elements, for example, and remove repeated `Space`s. diff --git a/pandoc.cabal b/pandoc.cabal index cd04a2faf..c47273c57 100644 --- a/pandoc.cabal +++ b/pandoc.cabal @@ -360,6 +360,7 @@ Executable pandoc extensible-exceptions >= 0.1 && < 0.2, highlighting-kate >= 0.5.5 && < 0.6, HTTP >= 4000.0.5 && < 4000.3, + process >= 1 && < 1.2, citeproc-hs >= 0.3.7 && < 0.4 if impl(ghc >= 7.0.1) Ghc-Options: -rtsopts -with-rtsopts=-K16m -Wall -fno-warn-unused-do-bind diff --git a/pandoc.hs b/pandoc.hs index 79bade221..94d206103 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -43,6 +43,7 @@ import System.Environment ( getArgs, getProgName ) import System.Exit ( exitWith, ExitCode (..) ) import System.FilePath import System.Console.GetOpt +import System.Process (readProcess) import Data.Char ( toLower ) import Data.List ( intercalate, isPrefixOf, sort ) import System.Directory ( getAppUserDataDirectory, doesFileExist, findExecutable ) @@ -53,6 +54,7 @@ import Control.Exception.Extensible ( throwIO ) import qualified Text.Pandoc.UTF8 as UTF8 import qualified Text.CSL as CSL import Control.Monad (when, unless, liftM) +import Data.Foldable (foldrM) import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..)) import Network.URI (parseURI, isURI, URI(..)) import qualified Data.ByteString.Lazy as B @@ -86,6 +88,12 @@ wrapWords indent c = wrap' (c - indent) (c - indent) isTextFormat :: String -> Bool isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub","epub3"] +externalFilter :: FilePath -> Pandoc -> IO Pandoc +externalFilter f d = E.catch + (readJSON def `fmap` readProcess f [] (writeJSON def d)) + (\e -> let _ = (e :: E.SomeException) + in err 83 $ "Error running filter `" ++ f ++ "'") + -- | Data structure for command line options. data Opt = Opt { optTabStop :: Int -- ^ Number of spaces per tab @@ -272,6 +280,13 @@ options = "STRING") "" -- "Classes (whitespace- or comma-separated) to use for indented code-blocks" + , Option "" ["filter"] + (ReqArg + (\arg opt -> return opt { optPlugins = externalFilter arg : + optPlugins opt }) + "PROGRAM") + "" -- "External JSON filter" + , Option "" ["normalize"] (NoArg (\opt -> return opt { optTransforms = @@ -876,6 +891,7 @@ main = do , optReferenceLinks = referenceLinks , optWrapText = wrap , optColumns = columns + , optPlugins = plugins , optEmailObfuscation = obfuscationMethod , optIdentifierPrefix = idPrefix , optIndentedCodeClasses = codeBlockClasses @@ -1099,6 +1115,7 @@ main = do reader readerOpts let doc0 = foldr ($) doc transforms + doc1 <- foldrM ($) doc0 plugins let writeBinary :: B.ByteString -> IO () writeBinary = B.writeFile (UTF8.encodePath outputFile) @@ -1109,15 +1126,15 @@ main = do case getWriter writerName' of Left e -> err 9 e - Right (IOStringWriter f) -> f writerOptions doc0 >>= writerFn outputFile - Right (IOByteStringWriter f) -> f writerOptions doc0 >>= writeBinary + Right (IOStringWriter f) -> f writerOptions doc1 >>= writerFn outputFile + Right (IOByteStringWriter f) -> f writerOptions doc1 >>= writeBinary Right (PureStringWriter f) | pdfOutput -> do - res <- makePDF latexEngine f writerOptions doc0 + res <- makePDF latexEngine f writerOptions doc1 case res of Right pdf -> writeBinary pdf Left err' -> err 43 $ UTF8.toStringLazy err' - | otherwise -> selfcontain (f writerOptions doc0 ++ + | otherwise -> selfcontain (f writerOptions doc1 ++ ['\n' | not standalone']) >>= writerFn outputFile . handleEntities where htmlFormat = writerName' `elem` -- cgit v1.2.3 From 2e5edbb27837372f658b1abbe05371be57415847 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Tue, 23 Jul 2013 23:17:07 -0700 Subject: Revert "Added `--filter` option." This reverts commit 85dacbb282f34dc45a810d7c7b5a2d639b70e1f5. --- README | 8 -------- pandoc.cabal | 1 - pandoc.hs | 25 ++++--------------------- 3 files changed, 4 insertions(+), 30 deletions(-) (limited to 'README') diff --git a/README b/README index 83873a92e..d9b003344 100644 --- a/README +++ b/README @@ -259,14 +259,6 @@ Reader options require different kinds of images. Currently this option only affects the markdown and LaTeX readers. -`--filter=`*PATH* -: Specify an executable to be used as a filter transforming the - Pandoc AST after the input is parsed and before the output is - written. The executable should read JSON from stdin and write - JSON to stdout. The JSON must be formatted like pandoc's own - JSON input and output. Filters may be most easily created in Haskell, - using the utility function `toJsonFilter` from `Text.Pandoc`. - `--normalize` : Normalize the document after reading: merge adjacent `Str` or `Emph` elements, for example, and remove repeated `Space`s. diff --git a/pandoc.cabal b/pandoc.cabal index c47273c57..cd04a2faf 100644 --- a/pandoc.cabal +++ b/pandoc.cabal @@ -360,7 +360,6 @@ Executable pandoc extensible-exceptions >= 0.1 && < 0.2, highlighting-kate >= 0.5.5 && < 0.6, HTTP >= 4000.0.5 && < 4000.3, - process >= 1 && < 1.2, citeproc-hs >= 0.3.7 && < 0.4 if impl(ghc >= 7.0.1) Ghc-Options: -rtsopts -with-rtsopts=-K16m -Wall -fno-warn-unused-do-bind diff --git a/pandoc.hs b/pandoc.hs index 94d206103..79bade221 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -43,7 +43,6 @@ import System.Environment ( getArgs, getProgName ) import System.Exit ( exitWith, ExitCode (..) ) import System.FilePath import System.Console.GetOpt -import System.Process (readProcess) import Data.Char ( toLower ) import Data.List ( intercalate, isPrefixOf, sort ) import System.Directory ( getAppUserDataDirectory, doesFileExist, findExecutable ) @@ -54,7 +53,6 @@ import Control.Exception.Extensible ( throwIO ) import qualified Text.Pandoc.UTF8 as UTF8 import qualified Text.CSL as CSL import Control.Monad (when, unless, liftM) -import Data.Foldable (foldrM) import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..)) import Network.URI (parseURI, isURI, URI(..)) import qualified Data.ByteString.Lazy as B @@ -88,12 +86,6 @@ wrapWords indent c = wrap' (c - indent) (c - indent) isTextFormat :: String -> Bool isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub","epub3"] -externalFilter :: FilePath -> Pandoc -> IO Pandoc -externalFilter f d = E.catch - (readJSON def `fmap` readProcess f [] (writeJSON def d)) - (\e -> let _ = (e :: E.SomeException) - in err 83 $ "Error running filter `" ++ f ++ "'") - -- | Data structure for command line options. data Opt = Opt { optTabStop :: Int -- ^ Number of spaces per tab @@ -280,13 +272,6 @@ options = "STRING") "" -- "Classes (whitespace- or comma-separated) to use for indented code-blocks" - , Option "" ["filter"] - (ReqArg - (\arg opt -> return opt { optPlugins = externalFilter arg : - optPlugins opt }) - "PROGRAM") - "" -- "External JSON filter" - , Option "" ["normalize"] (NoArg (\opt -> return opt { optTransforms = @@ -891,7 +876,6 @@ main = do , optReferenceLinks = referenceLinks , optWrapText = wrap , optColumns = columns - , optPlugins = plugins , optEmailObfuscation = obfuscationMethod , optIdentifierPrefix = idPrefix , optIndentedCodeClasses = codeBlockClasses @@ -1115,7 +1099,6 @@ main = do reader readerOpts let doc0 = foldr ($) doc transforms - doc1 <- foldrM ($) doc0 plugins let writeBinary :: B.ByteString -> IO () writeBinary = B.writeFile (UTF8.encodePath outputFile) @@ -1126,15 +1109,15 @@ main = do case getWriter writerName' of Left e -> err 9 e - Right (IOStringWriter f) -> f writerOptions doc1 >>= writerFn outputFile - Right (IOByteStringWriter f) -> f writerOptions doc1 >>= writeBinary + Right (IOStringWriter f) -> f writerOptions doc0 >>= writerFn outputFile + Right (IOByteStringWriter f) -> f writerOptions doc0 >>= writeBinary Right (PureStringWriter f) | pdfOutput -> do - res <- makePDF latexEngine f writerOptions doc1 + res <- makePDF latexEngine f writerOptions doc0 case res of Right pdf -> writeBinary pdf Left err' -> err 43 $ UTF8.toStringLazy err' - | otherwise -> selfcontain (f writerOptions doc1 ++ + | otherwise -> selfcontain (f writerOptions doc0 ++ ['\n' | not standalone']) >>= writerFn outputFile . handleEntities where htmlFormat = writerName' `elem` -- cgit v1.2.3 From a97f39c12e7b47a272575b69ad4cdd38966c043e Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Fri, 26 Jul 2013 12:40:56 -0700 Subject: Beamer: add allowframebreaks to slide if set in header classes. It's recommended that your bibliography slide have this attribute: # References {.allowframebreaks} This causes multiple slides to be created if necessary, depending on the length of the bibliography. --- README | 10 ++++++++++ src/Text/Pandoc/Writers/LaTeX.hs | 19 +++++++++++-------- 2 files changed, 21 insertions(+), 8 deletions(-) (limited to 'README') diff --git a/README b/README index d9b003344..4895f0f52 100644 --- a/README +++ b/README @@ -2674,6 +2674,16 @@ using the `-V` option: pandoc -t beamer habits.txt -V theme:Warsaw -o habits.pdf +Note that header attributes will turn into slide attributes +(on a `
` or `
`) in HTML slide formats, allowing you +to style individual slides. In Beamer, the only header attribute +that affects slides is the `allowframebreaks` class, which sets the +`allowframebreaks` option, causing multiple slides to be created +if the content overfills the frame. This is recommended especially for +bibliographies: + + # References {.allowframebreaks} + Literate Haskell support ======================== diff --git a/src/Text/Pandoc/Writers/LaTeX.hs b/src/Text/Pandoc/Writers/LaTeX.hs index 06a04ade2..aa5bfa623 100644 --- a/src/Text/Pandoc/Writers/LaTeX.hs +++ b/src/Text/Pandoc/Writers/LaTeX.hs @@ -232,7 +232,7 @@ toSlides bs = do elementToBeamer :: Int -> Element -> State WriterState [Block] elementToBeamer _slideLevel (Blk b) = return [b] -elementToBeamer slideLevel (Sec lvl _num (ident,classes,_) tit elts) +elementToBeamer slideLevel (Sec lvl _num (ident,classes,kvs) tit elts) | lvl > slideLevel = do bs <- concat `fmap` mapM (elementToBeamer slideLevel) elts return $ Para ( RawInline "latex" "\\begin{block}{" @@ -240,7 +240,7 @@ elementToBeamer slideLevel (Sec lvl _num (ident,classes,_) tit elts) : bs ++ [RawBlock "latex" "\\end{block}"] | lvl < slideLevel = do bs <- concat `fmap` mapM (elementToBeamer slideLevel) elts - return $ (Header lvl (ident,classes,[]) tit) : bs + return $ (Header lvl (ident,classes,kvs) tit) : bs | otherwise = do -- lvl == slideLevel -- note: [fragile] is required or verbatim breaks let hasCodeBlock (CodeBlock _ _) = [True] @@ -248,17 +248,20 @@ elementToBeamer slideLevel (Sec lvl _num (ident,classes,_) tit elts) let hasCode (Code _ _) = [True] hasCode _ = [] opts <- gets stOptions - let fragile = if not $ null $ queryWith hasCodeBlock elts ++ + let fragile = not $ null $ queryWith hasCodeBlock elts ++ if writerListings opts then queryWith hasCode elts else [] - then "[fragile]" - else "" - let slideStart = Para $ RawInline "latex" ("\\begin{frame}" ++ fragile) : + let allowframebreaks = "allowframebreaks" `elem` classes + let optionslist = ["fragile" | fragile] ++ + ["allowframebreaks" | allowframebreaks] + let options = if null optionslist + then "" + else "[" ++ intercalate "," optionslist ++ "]" + let slideStart = Para $ RawInline "latex" ("\\begin{frame}" ++ options) : if tit == [Str "\0"] -- marker for hrule then [] - else (RawInline "latex" "\\frametitle{") : tit ++ - [RawInline "latex" "}"] + else (RawInline "latex" "{") : tit ++ [RawInline "latex" "}"] let slideEnd = RawBlock "latex" "\\end{frame}" -- now carve up slide into blocks if there are sections inside bs <- concat `fmap` mapM (elementToBeamer slideLevel) elts -- cgit v1.2.3 From 1dde96e41879dd2ae39e8e79f8c77914f1885947 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Mon, 29 Jul 2013 08:38:29 -0700 Subject: Document that `fancy_lists` turns on `#` as list marker. --- README | 6 ++++++ 1 file changed, 6 insertions(+) (limited to 'README') diff --git a/README b/README index 4895f0f52..dbb0150d9 100644 --- a/README +++ b/README @@ -1342,6 +1342,12 @@ capital letter with a period, by at least two spaces.[^2] (C\) 2007 Joe Smith +The `fancy_lists` extension also allows '`#`' to be used as an +ordered list marker in place of a numeral: + + #. one + #. two + **Extension: `startnum`** Pandoc also pays attention to the type of list marker used, and to the -- cgit v1.2.3 From 208a9e076d577fb5c275ec797b539305300f6a95 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Tue, 6 Aug 2013 18:07:27 -0700 Subject: Beamer template: Added fonttheme variable. --- README | 6 +++--- data/templates | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) (limited to 'README') diff --git a/README b/README index dbb0150d9..bc5d5619d 100644 --- a/README +++ b/README @@ -761,7 +761,7 @@ as `title`, `author`, and `date`) as well as the following: `revealjs-url` : base URL for reveal.js documents (defaults to `reveal.js`) `theme` -: reveal.js theme +: reveal.js or LaTeX beamer theme `transition` : reveal.js transition `fontsize` @@ -777,10 +777,10 @@ as `title`, `author`, and `date`) as well as the following: `mainfont`, `sansfont`, `monofont`, `mathfont` : fonts for LaTeX documents (works only with xelatex and lualatex) -`theme` -: theme for LaTeX beamer documents `colortheme` : colortheme for LaTeX beamer documents +`fonttheme` +: fonttheme for LaTeX beamer documents `linkcolor` : color for internal links in LaTeX documents (`red`, `green`, `magenta`, `cyan`, `blue`, `black`) diff --git a/data/templates b/data/templates index 120bed772..c27f59c01 160000 --- a/data/templates +++ b/data/templates @@ -1 +1 @@ -Subproject commit 120bed772b4a235a0d17ea4560a682e23ac293ac +Subproject commit c27f59c010b0468f01b710cdf3a3c04a450a03e7 -- cgit v1.2.3 From 2677e8466311e2becdd7f65f1f6c23d559db14aa Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Thu, 8 Aug 2013 11:09:00 -0700 Subject: Revert "Revert "Added `--filter` option."" This reverts commit 2e5edbb27837372f658b1abbe05371be57415847. --- README | 8 ++++++++ pandoc.cabal | 1 + pandoc.hs | 25 +++++++++++++++++++++---- 3 files changed, 30 insertions(+), 4 deletions(-) (limited to 'README') diff --git a/README b/README index bc5d5619d..e85ca1905 100644 --- a/README +++ b/README @@ -259,6 +259,14 @@ Reader options require different kinds of images. Currently this option only affects the markdown and LaTeX readers. +`--filter=`*PATH* +: Specify an executable to be used as a filter transforming the + Pandoc AST after the input is parsed and before the output is + written. The executable should read JSON from stdin and write + JSON to stdout. The JSON must be formatted like pandoc's own + JSON input and output. Filters may be most easily created in Haskell, + using the utility function `toJsonFilter` from `Text.Pandoc`. + `--normalize` : Normalize the document after reading: merge adjacent `Str` or `Emph` elements, for example, and remove repeated `Space`s. diff --git a/pandoc.cabal b/pandoc.cabal index 19f8c14ef..a8dd528ad 100644 --- a/pandoc.cabal +++ b/pandoc.cabal @@ -352,6 +352,7 @@ Executable pandoc extensible-exceptions >= 0.1 && < 0.2, highlighting-kate >= 0.5.5 && < 0.6, HTTP >= 4000.0.5 && < 4000.3, + process >= 1 && < 1.2, citeproc-hs >= 0.3.7 && < 0.4 Ghc-Options: -rtsopts -with-rtsopts=-K16m -Wall -fno-warn-unused-do-bind Ghc-Prof-Options: -auto-all -caf-all -rtsopts -with-rtsopts=-K16m diff --git a/pandoc.hs b/pandoc.hs index 79bade221..94d206103 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -43,6 +43,7 @@ import System.Environment ( getArgs, getProgName ) import System.Exit ( exitWith, ExitCode (..) ) import System.FilePath import System.Console.GetOpt +import System.Process (readProcess) import Data.Char ( toLower ) import Data.List ( intercalate, isPrefixOf, sort ) import System.Directory ( getAppUserDataDirectory, doesFileExist, findExecutable ) @@ -53,6 +54,7 @@ import Control.Exception.Extensible ( throwIO ) import qualified Text.Pandoc.UTF8 as UTF8 import qualified Text.CSL as CSL import Control.Monad (when, unless, liftM) +import Data.Foldable (foldrM) import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..)) import Network.URI (parseURI, isURI, URI(..)) import qualified Data.ByteString.Lazy as B @@ -86,6 +88,12 @@ wrapWords indent c = wrap' (c - indent) (c - indent) isTextFormat :: String -> Bool isTextFormat s = takeWhile (`notElem` "+-") s `notElem` ["odt","docx","epub","epub3"] +externalFilter :: FilePath -> Pandoc -> IO Pandoc +externalFilter f d = E.catch + (readJSON def `fmap` readProcess f [] (writeJSON def d)) + (\e -> let _ = (e :: E.SomeException) + in err 83 $ "Error running filter `" ++ f ++ "'") + -- | Data structure for command line options. data Opt = Opt { optTabStop :: Int -- ^ Number of spaces per tab @@ -272,6 +280,13 @@ options = "STRING") "" -- "Classes (whitespace- or comma-separated) to use for indented code-blocks" + , Option "" ["filter"] + (ReqArg + (\arg opt -> return opt { optPlugins = externalFilter arg : + optPlugins opt }) + "PROGRAM") + "" -- "External JSON filter" + , Option "" ["normalize"] (NoArg (\opt -> return opt { optTransforms = @@ -876,6 +891,7 @@ main = do , optReferenceLinks = referenceLinks , optWrapText = wrap , optColumns = columns + , optPlugins = plugins , optEmailObfuscation = obfuscationMethod , optIdentifierPrefix = idPrefix , optIndentedCodeClasses = codeBlockClasses @@ -1099,6 +1115,7 @@ main = do reader readerOpts let doc0 = foldr ($) doc transforms + doc1 <- foldrM ($) doc0 plugins let writeBinary :: B.ByteString -> IO () writeBinary = B.writeFile (UTF8.encodePath outputFile) @@ -1109,15 +1126,15 @@ main = do case getWriter writerName' of Left e -> err 9 e - Right (IOStringWriter f) -> f writerOptions doc0 >>= writerFn outputFile - Right (IOByteStringWriter f) -> f writerOptions doc0 >>= writeBinary + Right (IOStringWriter f) -> f writerOptions doc1 >>= writerFn outputFile + Right (IOByteStringWriter f) -> f writerOptions doc1 >>= writeBinary Right (PureStringWriter f) | pdfOutput -> do - res <- makePDF latexEngine f writerOptions doc0 + res <- makePDF latexEngine f writerOptions doc1 case res of Right pdf -> writeBinary pdf Left err' -> err 43 $ UTF8.toStringLazy err' - | otherwise -> selfcontain (f writerOptions doc0 ++ + | otherwise -> selfcontain (f writerOptions doc1 ++ ['\n' | not standalone']) >>= writerFn outputFile . handleEntities where htmlFormat = writerName' `elem` -- cgit v1.2.3 From bd73d73a28acc2863bd52bdc6f0f9d850fa34d84 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Tue, 13 Aug 2013 18:25:20 -0700 Subject: Removed `--print-sample-lua-writer`, added `--print-default-data-file`. Closes #943. --- README | 7 +++---- pandoc.hs | 19 ++++++++++--------- 2 files changed, 13 insertions(+), 13 deletions(-) (limited to 'README') diff --git a/README b/README index e85ca1905..e5de97556 100644 --- a/README +++ b/README @@ -311,9 +311,8 @@ General writer options : Print the default template for an output *FORMAT*. (See `-t` for a list of possible *FORMAT*s.) -`--print-sample-lua-writer` -: Print a sample lua custom writer (see [Custom writers](#custom-writers), - below. +`--print-default-data-file=`*FILE* +: Print a default data file. `--no-wrap` : Disable text wrapping in output. By default, text is wrapped @@ -2759,7 +2758,7 @@ Creating a custom writer requires writing a lua function for each possible element in a pandoc document. To get a documented example which you can modify according to your needs, do - pandoc --print-sample-lua-writer + pandoc --print-default-data-file sample.lua Authors ======= diff --git a/pandoc.hs b/pandoc.hs index 81672e16c..1c0a49a73 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -33,8 +33,8 @@ module Main where import Text.Pandoc import Text.Pandoc.PDF (makePDF) import Text.Pandoc.Readers.LaTeX (handleIncludes) -import Text.Pandoc.Shared ( tabFilter, readDataFileUTF8, safeRead, - headerShift, normalize, err, warn ) +import Text.Pandoc.Shared ( tabFilter, readDataFileUTF8, readDataFile, + safeRead, headerShift, normalize, err, warn ) import Text.Pandoc.XML ( toEntities, fromEntities ) import Text.Pandoc.SelfContained ( makeSelfContained ) import Text.Pandoc.Process (pipeProcess) @@ -58,6 +58,7 @@ import Data.Foldable (foldrM) import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..)) import Network.URI (parseURI, isURI, URI(..)) import qualified Data.ByteString.Lazy as B +import qualified Data.ByteString as BS import Text.CSL.Reference (Reference(..)) import Data.Aeson (eitherDecode', encode) @@ -347,13 +348,13 @@ options = "FORMAT") "" -- "Print default template for FORMAT" - , Option "" ["print-sample-lua-writer"] - (NoArg - (\_ -> do - sample <- readDataFileUTF8 Nothing "sample.lua" - UTF8.hPutStr stdout sample - exitWith ExitSuccess)) - "" -- "Print sample lua custom writer" + , Option "" ["print-default-data-file"] + (ReqArg + (\arg _ -> do + readDataFile Nothing arg >>= BS.hPutStr stdout + exitWith ExitSuccess) + "FILE") + "" -- "Print default data file" , Option "" ["no-wrap"] (NoArg -- cgit v1.2.3 From 89a7703260703599a033be16e1581a0494326c2b Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Fri, 16 Aug 2013 13:22:27 -0700 Subject: Shared: Changed stringify so it ignores notes. Also documented this in README. --- README | 1 + src/Text/Pandoc/Shared.hs | 6 +++++- 2 files changed, 6 insertions(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index e5de97556..c25d611d6 100644 --- a/README +++ b/README @@ -980,6 +980,7 @@ automatically assigned a unique identifier based on the header text. To derive the identifier from the header text, - Remove all formatting, links, etc. + - Remove all footnotes. - Remove all punctuation, except underscores, hyphens, and periods. - Replace all spaces and newlines with hyphens. - Convert all alphabetic characters to lowercase. diff --git a/src/Text/Pandoc/Shared.hs b/src/Text/Pandoc/Shared.hs index 72b467da5..eef150351 100644 --- a/src/Text/Pandoc/Shared.hs +++ b/src/Text/Pandoc/Shared.hs @@ -384,8 +384,10 @@ consolidateInlines (x : xs) = x : consolidateInlines xs consolidateInlines [] = [] -- | Convert list of inlines to a string with formatting removed. +-- Footnotes are skipped (since we don't want their contents in link +-- labels). stringify :: [Inline] -> String -stringify = query go +stringify = query go . walk deNote where go :: Inline -> [Char] go Space = " " go (Str x) = x @@ -393,6 +395,8 @@ stringify = query go go (Math _ x) = x go LineBreak = " " go _ = "" + deNote (Note _) = Str "" + deNote x = x -- | Change final list item from @Para@ to @Plain@ if the list contains -- no other @Para@ blocks. -- cgit v1.2.3 From 05cc3a5c2182072ba93ffe4ac8b6661c098202e7 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 18 Aug 2013 15:58:08 -0700 Subject: Improved `--filter` documentation. --- README | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) (limited to 'README') diff --git a/README b/README index c25d611d6..c1429aec3 100644 --- a/README +++ b/README @@ -259,13 +259,30 @@ Reader options require different kinds of images. Currently this option only affects the markdown and LaTeX readers. -`--filter=`*PATH* +`--filter=`*EXECUTABLE* : Specify an executable to be used as a filter transforming the Pandoc AST after the input is parsed and before the output is written. The executable should read JSON from stdin and write JSON to stdout. The JSON must be formatted like pandoc's own - JSON input and output. Filters may be most easily created in Haskell, - using the utility function `toJsonFilter` from `Text.Pandoc`. + JSON input and output. The name of the output format will be + passed to the filter as the first argument. Hence, + + pandoc --filter ./caps.py -t latex + + is equivalent to + + pandoc -t json | ./caps.py latex | pandoc -f json -t latex + + The latter form may be useful for debugging filters. + + Filters may be written in any language. `Text.Pandoc.JSON` + exports `toJSONFilter` to facilitate writing filters in Haskell. + Those who would prefer to write filters in pandoc can use the + module `pandoc.py`: see + for the module and several examples. Note that the *EXECUTABLE* + will be sought in the user's `PATH`, and not in the working directory, + if no directory is provided. If you want to run a script in the + working directory, preface the filename with `./`. `--normalize` : Normalize the document after reading: merge adjacent -- cgit v1.2.3 From 0e2605ffdf69b7a6a7c942a986dec4283a886e82 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 18 Aug 2013 18:39:04 -0700 Subject: Allow multiple YAML metadata blocks in document. --- README | 27 +++++---- src/Text/Pandoc/Readers/Markdown.hs | 107 +++++++++++++++++++----------------- 2 files changed, 73 insertions(+), 61 deletions(-) (limited to 'README') diff --git a/README b/README index c1429aec3..7a2b01f49 100644 --- a/README +++ b/README @@ -1815,14 +1815,21 @@ YAML metadata block **Extension: `yaml_metadata_block`** -If the file begins with a YAML object, delimited by a line of three -hyphens (`---`) at the top and a line of three hyphens (`---`) or three -dots (`...`) at the bottom, metadata will be taken from the fields -of the YAML object. Metadata can contain lists and objects (nested -arbitrarily), but all string scalars will be interpreted as markdown. - -Fields with names ending in an underscore will be ignored by -pandoc. (They may be given a role by external processors.) +A YAML metadata block is a valid YAML object, delimited by a line of three +hyphens (`---`) at the top and a line of three hyphens (`---`) or three dots +(`...`) at the bottom. A YAML metadata block may occur anywhere in the +document, but if it is not at the beginning, it must be preceded by a blank +line. + +Metadata will be taken from the fields of the YAML object and added to any +existing document metadata. Metadata can contain lists and objects (nested +arbitrarily), but all string scalars will be interpreted as markdown. Fields +with names ending in an underscore will be ignored by pandoc. (They may be +given a role by external processors.) + +A document may contain multiple metadata blocks. The metadata fields will +be combined through a *left-biased union*: if two metadata blocks attempt +to set the same field, the value from the first block will be taken. Note that YAML escaping rules must be followed. Thus, for example, if a title contains a colon, it must be quoted. The pipe character @@ -1844,8 +1851,8 @@ when the field contains blank lines: It consists of two paragraphs. ... -Template variables will be set from the metadata. Thus, for example, -in writing HTML, the variable `abstract` will be set to the HTML +Template variables will be set automatically from the metadata. Thus, for +example, in writing HTML, the variable `abstract` will be set to the HTML equivalent of the markdown in the `abstract` field:

This is the abstract.

diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs index 535fc02c6..a653c2e98 100644 --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -203,13 +203,10 @@ dateLine = try $ do skipSpaces trimInlinesF . mconcat <$> manyTill inline newline -titleBlock :: MarkdownParser (F (Pandoc -> Pandoc)) -titleBlock = pandocTitleBlock - <|> yamlTitleBlock - <|> mmdTitleBlock - <|> return (return id) +titleBlock :: MarkdownParser () +titleBlock = pandocTitleBlock <|> mmdTitleBlock -pandocTitleBlock :: MarkdownParser (F (Pandoc -> Pandoc)) +pandocTitleBlock :: MarkdownParser () pandocTitleBlock = try $ do guardEnabled Ext_pandoc_title_block lookAhead (char '%') @@ -217,16 +214,18 @@ pandocTitleBlock = try $ do author <- option (return []) authorsLine date <- option mempty dateLine optional blanklines - return $ do - title' <- title - author' <- author - date' <- date - return $ if B.isNull title' then id else B.setMeta "title" title' - . if null author' then id else B.setMeta "author" author' - . if B.isNull date' then id else B.setMeta "date" date' - -yamlTitleBlock :: MarkdownParser (F (Pandoc -> Pandoc)) -yamlTitleBlock = try $ do + let meta' = do title' <- title + author' <- author + date' <- date + return $ + ( if B.isNull title' then id else B.setMeta "title" title' + . if null author' then id else B.setMeta "author" author' + . if B.isNull date' then id else B.setMeta "date" date' ) + nullMeta + updateState $ \st -> st{ stateMeta' = stateMeta' st <> meta' } + +yamlMetaBlock :: MarkdownParser (F Blocks) +yamlMetaBlock = try $ do guardEnabled Ext_yaml_metadata_block pos <- getPosition string "---" @@ -236,33 +235,39 @@ yamlTitleBlock = try $ do let rawYaml = unlines ("---" : (rawYamlLines ++ ["..."])) optional blanklines opts <- stateOptions <$> getState - case Yaml.decodeEither' $ UTF8.fromString rawYaml of - Right (Yaml.Object hashmap) -> return $ return $ - H.foldrWithKey (\k v f -> - if ignorable k - then f - else B.setMeta (T.unpack k) (yamlToMeta opts v) . f) - id hashmap - Right Yaml.Null -> return $ return id - Right _ -> do - addWarning (Just pos) "YAML header is not an object" - return $ return id - Left err' -> do - case err' of - InvalidYaml (Just YamlParseException{ - yamlProblem = problem - , yamlContext = _ctxt - , yamlProblemMark = Yaml.YamlMark { - yamlLine = yline - , yamlColumn = ycol - }}) -> - addWarning (Just $ setSourceLine - (setSourceColumn pos (sourceColumn pos + ycol)) - (sourceLine pos + 1 + yline)) - $ "Could not parse YAML header: " ++ problem - _ -> addWarning (Just pos) - $ "Could not parse YAML header: " ++ show err' - return $ return id + meta' <- case Yaml.decodeEither' $ UTF8.fromString rawYaml of + Right (Yaml.Object hashmap) -> return $ return $ + H.foldrWithKey (\k v m -> + if ignorable k + then m + else B.setMeta (T.unpack k) + (yamlToMeta opts v) m) + nullMeta hashmap + Right Yaml.Null -> return $ return nullMeta + Right _ -> do + addWarning (Just pos) "YAML header is not an object" + return $ return nullMeta + Left err' -> do + case err' of + InvalidYaml (Just YamlParseException{ + yamlProblem = problem + , yamlContext = _ctxt + , yamlProblemMark = Yaml.YamlMark { + yamlLine = yline + , yamlColumn = ycol + }}) -> + addWarning (Just $ setSourceLine + (setSourceColumn pos + (sourceColumn pos + ycol)) + (sourceLine pos + 1 + yline)) + $ "Could not parse YAML header: " ++ + problem + _ -> addWarning (Just pos) + $ "Could not parse YAML header: " ++ + show err' + return $ return nullMeta + updateState $ \st -> st{ stateMeta' = stateMeta' st <> meta' } + return mempty -- ignore fields ending with _ ignorable :: Text -> Bool @@ -295,13 +300,13 @@ yamlToMeta _ _ = MetaString "" stopLine :: MarkdownParser () stopLine = try $ (string "---" <|> string "...") >> blankline >> return () -mmdTitleBlock :: MarkdownParser (F (Pandoc -> Pandoc)) +mmdTitleBlock :: MarkdownParser () mmdTitleBlock = try $ do guardEnabled Ext_mmd_title_block kvPairs <- many1 kvPair blanklines - return $ return $ \(Pandoc m bs) -> - Pandoc (foldl (\m' (k,v) -> addMetaField k v m') m kvPairs) bs + updateState $ \st -> st{ stateMeta' = stateMeta' st <> + return (Meta $ M.fromList kvPairs) } kvPair :: MarkdownParser (String, MetaValue) kvPair = try $ do @@ -318,15 +323,14 @@ parseMarkdown = do updateState $ \state -> state { stateOptions = let oldOpts = stateOptions state in oldOpts{ readerParseRaw = True } } - titleTrans <- option (return id) titleBlock + optional titleBlock blocks <- parseBlocks st <- getState + let meta = runF (stateMeta' st) st + let Pandoc _ bs = B.doc $ runF blocks st mbsty <- getOption readerCitationStyle refs <- getOption readerReferences - return $ processBiblio mbsty refs - $ runF titleTrans st - $ B.doc - $ runF blocks st + return $ processBiblio mbsty refs $ Pandoc meta bs addWarning :: Maybe SourcePos -> String -> MarkdownParser () addWarning mbpos msg = @@ -442,6 +446,7 @@ parseBlocks = mconcat <$> manyTill block eof block :: MarkdownParser (F Blocks) block = choice [ mempty <$ blanklines , codeBlockFenced + , yamlMetaBlock , guardEnabled Ext_latex_macros *> (macro >>= return . return) , header , lhsCodeBlock -- cgit v1.2.3 From deb59b62354e38df9c85ce6985e5c28dd2301ee7 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sat, 24 Aug 2013 22:27:08 -0700 Subject: Removed dependency on citeproc-hs. Going forward we'll use pandoc-citeproc, as an external filter. The `--bibliography`, `--csl`, and `--citation-abbreviation` fields have been removed. Instead one must include `bibliography`, `csl`, or `csl-abbrevs` fields in the document's YAML metadata. The filter can then be used as follows: pandoc --filter pandoc-citeproc The `Text.Pandoc.Biblio` module has been removed. Henceforth, `Text.CSL.Pandoc` from pandoc-citations can be used by library users. The Markdown and LaTeX readers now longer format bibliographies and citations. That must be done using `processCites` or `processCites'` from Text.CSL.Pandoc. All bibliography-related fields have been removed from `ReaderOptions` and `WriterOptions`: `writerBiblioFiles`, `readerReferences`, `readerCitationStyle`. API change. --- README | 105 ++++----- data/default.csl | 458 ------------------------------------ pandoc.cabal | 7 +- pandoc.hs | 68 +----- src/Text/Pandoc/Biblio.hs | 216 ----------------- src/Text/Pandoc/Options.hs | 7 - src/Text/Pandoc/Readers/LaTeX.hs | 5 +- src/Text/Pandoc/Readers/Markdown.hs | 5 +- src/Text/Pandoc/Writers/LaTeX.hs | 8 +- tests/Tests/Old.hs | 14 -- 10 files changed, 50 insertions(+), 843 deletions(-) delete mode 100644 data/default.csl delete mode 100644 src/Text/Pandoc/Biblio.hs (limited to 'README') diff --git a/README b/README index 7a2b01f49..f85e62e14 100644 --- a/README +++ b/README @@ -598,54 +598,6 @@ Options affecting specific writers Citation rendering ------------------ -`--bibliography=`*FILE* -: Specify bibliography database to be used in resolving - citations. The database type will be determined from the - extension of *FILE*, which may be `.mods` (MODS format), - `.bib` (BibLaTeX format, which will normally work for BibTeX - files as well), `.bibtex` (BibTeX format), - `.ris` (RIS format), `.enl` (EndNote format), - `.xml` (EndNote XML format), `.wos` (ISI format), - `.medline` (MEDLINE format), `.copac` (Copac format), - or `.json` (citeproc JSON). If you want to use multiple - bibliographies, just use this option repeatedly. - -`--csl=`*FILE* -: Specify [CSL] style to be used in formatting citations and - the bibliography. If *FILE* is not found, pandoc will look - for it in - - $HOME/.csl - - in unix, - - C:\Documents And Settings\USERNAME\Application Data\csl - - in Windows XP, and - - C:\Users\USERNAME\AppData\Roaming\csl - - in Windows 7. If the `--csl` option is not specified, pandoc - will use a default style: either `default.csl` in the - user data directory (see `--data-dir`), or, if that is - not present, the Chicago author-date style. - -`--citation-abbreviations=`*FILE* -: Specify a file containing abbreviations for journal titles and - other bibliographic fields (indicated by setting `form="short"` - in the CSL node for the field). The format is described at - . - Here is a short example: - - { "default": { - "container-title": { - "Lloyd's Law Reports": "Lloyd's Rep", - "Estates Gazette": "EG", - "Scots Law Times": "SLT" - } - } - } - `--natbib` : Use natbib for citations in LaTeX output. @@ -2378,9 +2330,14 @@ Citations **Extension: `citations`** -Pandoc can automatically generate citations and a bibliography in a number of -styles (using Andrea Rossato's `hs-citeproc`). In order to use this feature, -you will need a bibliographic database in one of the following formats: +Using an external filter, `pandoc-citeproc`, pandoc can automatically generate +citations and a bibliography in a number of styles. Basic usage is + + pandoc --filter pandoc-citeproc myinput.txt + +In order to use this feature, you will need to specify a bibliography file +using the `bibliography` metadata field in a YAML metadata section. +The bibliography may have any of these formats: Format File extension ------------ -------------- @@ -2398,18 +2355,40 @@ you will need a bibliographic database in one of the following formats: Note that `.bib` can generally be used with both BibTeX and BibLaTeX files, but you can use `.bibtex` to force BibTeX. -You will need to specify the bibliography file using the `--bibliography` -command-line option (which may be repeated if you have several -bibliographies). - -By default, pandoc will use a Chicago author-date format for citations -and references. To use another style, you will need to use the -`--csl` option to specify a [CSL] 1.0 style file. A primer on -creating and modifying CSL styles can be found at -. -A repository of CSL styles can be found at -. -See also for easy browsing. +Alternatively you can use a `references` field in the document's YAML +metadata. This should include an array of YAML-encoded references, +for example: + + --- + references: + - id: fenner2012a + title: One-click science marketing + author: + - family: Fenner + given: Martin + container-title: Nature Materials + volume: 11 + URL: 'http://dx.doi.org/10.1038/nmat3283' + DOI: 10.1038/nmat3283 + issue: 4 + publisher: Nature Publishing Group + page: 261-263 + type: article-journal + issued: + year: 2012 + month: 3 + ... + +(The program `mods2yaml`, which comes with `pandoc-citeproc`, can help produce +these from a MODS reference collection.) + +By default, `pandoc-citeproc` will use a Chicago author-date format for +citations and references. To use another style, you will need to specify +a [CSL] 1.0 style file in the `csl` metadata field. A primer on creating and +modifying CSL styles can be found at +. A repository of CSL styles +can be found at . See also + for easy browsing. Citations go inside square brackets and are separated by semicolons. Each citation must have a key, composed of '@' + the citation diff --git a/data/default.csl b/data/default.csl deleted file mode 100644 index 83a70d0b5..000000000 --- a/data/default.csl +++ /dev/null @@ -1,458 +0,0 @@ - - diff --git a/pandoc.cabal b/pandoc.cabal index ac28ad068..0ab990a17 100644 --- a/pandoc.cabal +++ b/pandoc.cabal @@ -99,8 +99,6 @@ Data-Files: data/slideous/slideous.js, -- data for dzslides writer data/dzslides/template.html, - -- data for citeproc - data/default.csl, -- sample lua custom writer data/sample.lua -- documentation @@ -250,7 +248,6 @@ Library xml >= 1.3.12 && < 1.4, random >= 1 && < 1.1, extensible-exceptions >= 0.1 && < 0.2, - citeproc-hs >= 0.3.7 && < 0.4, pandoc-types >= 1.12 && < 1.13, aeson >= 0.6 && < 0.7, tagsoup >= 0.12.5 && < 0.14, @@ -323,7 +320,6 @@ Library Text.Pandoc.UTF8, Text.Pandoc.Templates, Text.Pandoc.XML, - Text.Pandoc.Biblio, Text.Pandoc.SelfContained, Text.Pandoc.Process Other-Modules: Text.Pandoc.Readers.Haddock.Lex, @@ -353,8 +349,7 @@ Executable pandoc extensible-exceptions >= 0.1 && < 0.2, highlighting-kate >= 0.5.5 && < 0.6, aeson >= 0.6 && < 0.7, - HTTP >= 4000.0.5 && < 4000.3, - citeproc-hs >= 0.3.7 && < 0.4 + HTTP >= 4000.0.5 && < 4000.3 Ghc-Options: -rtsopts -with-rtsopts=-K16m -Wall -fno-warn-unused-do-bind Ghc-Prof-Options: -auto-all -caf-all -rtsopts -with-rtsopts=-K16m if os(windows) diff --git a/pandoc.hs b/pandoc.hs index 8eed67544..6ad5694f1 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -35,7 +35,7 @@ import Text.Pandoc.PDF (makePDF) import Text.Pandoc.Readers.LaTeX (handleIncludes) import Text.Pandoc.Shared ( tabFilter, readDataFileUTF8, readDataFile, safeRead, headerShift, normalize, err, warn ) -import Text.Pandoc.XML ( toEntities, fromEntities ) +import Text.Pandoc.XML ( toEntities ) import Text.Pandoc.SelfContained ( makeSelfContained ) import Text.Pandoc.Process (pipeProcess) import Text.Highlighting.Kate ( languages, Style, tango, pygments, @@ -46,20 +46,18 @@ import System.FilePath import System.Console.GetOpt import Data.Char ( toLower ) import Data.List ( intercalate, isPrefixOf, sort ) -import System.Directory ( getAppUserDataDirectory, doesFileExist, findExecutable ) +import System.Directory ( getAppUserDataDirectory, findExecutable ) import System.IO ( stdout, stderr ) import System.IO.Error ( isDoesNotExistError ) import qualified Control.Exception as E import Control.Exception.Extensible ( throwIO ) import qualified Text.Pandoc.UTF8 as UTF8 -import qualified Text.CSL as CSL import Control.Monad (when, unless, liftM) import Data.Foldable (foldrM) import Network.HTTP (simpleHTTP, mkRequest, getResponseBody, RequestMethod(..)) import Network.URI (parseURI, isURI, URI(..)) import qualified Data.ByteString.Lazy as B import qualified Data.ByteString as BS -import Text.CSL.Reference (Reference(..)) import Data.Aeson (eitherDecode', encode) copyrightMessage :: String @@ -70,7 +68,7 @@ copyrightMessage = "\nCopyright (C) 2006-2013 John MacFarlane\n" ++ compileInfo :: String compileInfo = - "\nCompiled with citeproc-hs " ++ VERSION_citeproc_hs ++ ", texmath " ++ + "\nCompiled with texmath " ++ VERSION_texmath ++ ", highlighting-kate " ++ VERSION_highlighting_kate ++ ".\nSyntax highlighting is supported for the following languages:\n " ++ wrapWords 4 78 @@ -146,9 +144,6 @@ data Opt = Opt , optIndentedCodeClasses :: [String] -- ^ Default classes for indented code blocks , optDataDir :: Maybe FilePath , optCiteMethod :: CiteMethod -- ^ Method to output cites - , optBibliography :: [String] - , optCslFile :: Maybe FilePath - , optAbbrevsFile :: Maybe FilePath , optListings :: Bool -- ^ Use listings package for code blocks , optLaTeXEngine :: String -- ^ Program to use for latex -> pdf , optSlideLevel :: Maybe Int -- ^ Header level that creates slides @@ -203,9 +198,6 @@ defaultOpts = Opt , optIndentedCodeClasses = [] , optDataDir = Nothing , optCiteMethod = Citeproc - , optBibliography = [] - , optCslFile = Nothing - , optAbbrevsFile = Nothing , optListings = False , optLaTeXEngine = "pdflatex" , optSlideLevel = Nothing @@ -650,24 +642,6 @@ options = "PROGRAM") "" -- "Name of latex program to use in generating PDF" - , Option "" ["bibliography"] - (ReqArg - (\arg opt -> return opt { optBibliography = (optBibliography opt) ++ [arg] }) - "FILENAME") - "" - - , Option "" ["csl"] - (ReqArg - (\arg opt -> return opt { optCslFile = Just arg }) - "FILENAME") - "" - - , Option "" ["citation-abbreviations"] - (ReqArg - (\arg opt -> return opt { optAbbrevsFile = Just arg }) - "FILENAME") - "" - , Option "" ["natbib"] (NoArg (\opt -> return opt { optCiteMethod = Natbib })) @@ -904,9 +878,6 @@ main = do , optIdentifierPrefix = idPrefix , optIndentedCodeClasses = codeBlockClasses , optDataDir = mbDataDir - , optBibliography = reffiles - , optCslFile = mbCsl - , optAbbrevsFile = cslabbrevs , optCiteMethod = citeMethod , optListings = listings , optLaTeXEngine = latexEngine @@ -1007,36 +978,6 @@ main = do $ lines dztempl return $ ("dzslides-core", dzcore) : variables' else return variables' - - -- unescape reference ids, which may contain XML entities, so - -- that we can do lookups with regular string equality - let unescapeRefId ref = ref{ refId = fromEntities (refId ref) } - - refs <- mapM (\f -> E.catch (CSL.readBiblioFile f) - (\e -> let _ = (e :: E.SomeException) - in err 23 $ "Error reading bibliography `" ++ f ++ - "'" ++ "\n" ++ show e)) - reffiles >>= - return . map unescapeRefId . concat - - mbsty <- if citeMethod == Citeproc && not (null refs) - then do - csl <- CSL.parseCSL =<< - case mbCsl of - Nothing -> readDataFileUTF8 datadir - "default.csl" - Just cslfile -> do - exists <- doesFileExist cslfile - if exists - then UTF8.readFile cslfile - else do - csldir <- getAppUserDataDirectory "csl" - readDataFileUTF8 (Just csldir) - (replaceExtension cslfile "csl") - abbrevs <- maybe (return []) CSL.readJsonAbbrevFile cslabbrevs - return $ Just csl { CSL.styleAbbrevs = abbrevs } - else return Nothing - let sourceURL = case sources of [] -> Nothing (x:_) -> case parseURI x of @@ -1054,8 +995,6 @@ main = do , readerColumns = columns , readerTabStop = tabStop , readerOldDashes = oldDashes - , readerReferences = refs - , readerCitationStyle = mbsty , readerIndentedCodeClasses = codeBlockClasses , readerApplyMacros = not laTeXOutput , readerDefaultImageExtension = defaultImageExtension @@ -1069,7 +1008,6 @@ main = do writerHTMLMathMethod = mathMethod, writerIncremental = incremental, writerCiteMethod = citeMethod, - writerBiblioFiles = reffiles, writerIgnoreNotes = False, writerNumberSections = numberSections, writerNumberOffset = numberFrom, diff --git a/src/Text/Pandoc/Biblio.hs b/src/Text/Pandoc/Biblio.hs deleted file mode 100644 index 1c0975f11..000000000 --- a/src/Text/Pandoc/Biblio.hs +++ /dev/null @@ -1,216 +0,0 @@ -{-# LANGUAGE PatternGuards #-} -{- -Copyright (C) 2008 Andrea Rossato - -This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation; either version 2 of the License, or -(at your option) any later version. - -This program is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program; if not, write to the Free Software -Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA --} - -{- | - Module : Text.Pandoc.Biblio - Copyright : Copyright (C) 2008-2010 Andrea Rossato - License : GNU GPL, version 2 or above - - Maintainer : Andrea Rossato - Stability : alpha - Portability : portable --} - -module Text.Pandoc.Biblio ( processBiblio ) where - -import Data.List -import Data.Char ( isDigit, isPunctuation ) -import qualified Data.Map as M -import Text.CSL hiding ( Cite(..), Citation(..), endWithPunct ) -import qualified Text.CSL as CSL ( Cite(..) ) -import Text.Pandoc.Definition -import Text.Pandoc.Generic -import Text.Pandoc.Walk -import Text.Pandoc.Shared (stringify) -import Text.Parsec hiding (State) -import Control.Monad -import Control.Monad.State - --- | Process a 'Pandoc' document by adding citations formatted --- according to a CSL style, using 'citeproc' from citeproc-hs. -processBiblio :: Maybe Style -> [Reference] -> Pandoc -> Pandoc -processBiblio Nothing _ p = p -processBiblio _ [] p = p -processBiblio (Just style) r p = - let p' = evalState (bottomUpM setHash p) 1 - grps = query getCitation p' - result = citeproc procOpts style r (setNearNote style $ - map (map toCslCite) grps) - cits_map = M.fromList $ zip grps (citations result) - biblioList = map (renderPandoc' style) (bibliography result) - Pandoc m b = bottomUp mvPunct . deNote . topDown (processCite style cits_map) $ p' - (bs, lastb) = case reverse b of - x@(Header _ _ _) : xs -> (reverse xs, [x]) - _ -> (b, []) - in Pandoc m $ bs ++ [Div ("",["references"],[]) (lastb ++ biblioList)] - --- | Substitute 'Cite' elements with formatted citations. -processCite :: Style -> M.Map [Citation] [FormattedOutput] -> Inline -> Inline -processCite s cs (Cite t _) = - case M.lookup t cs of - Just (x:xs) - | isTextualCitation t && not (null xs) -> - let xs' = renderPandoc s xs - in if styleClass s == "note" - then Cite t (renderPandoc s [x] ++ [Note [Para xs']]) - else Cite t (renderPandoc s [x] ++ [Space | not (startWithPunct xs')] ++ xs') - | otherwise -> if styleClass s == "note" - then Cite t [Note [Para $ renderPandoc s (x:xs)]] - else Cite t (renderPandoc s (x:xs)) - _ -> Strong [Str "???"] -- TODO raise error instead? -processCite _ _ x = x - -isNote :: Inline -> Bool -isNote (Note _) = True -isNote (Cite _ [Note _]) = True -isNote _ = False - -mvPunct :: [Inline] -> [Inline] -mvPunct (Space : Space : xs) = Space : xs -mvPunct (Space : x : ys) | isNote x, startWithPunct ys = - Str (headInline ys) : x : tailFirstInlineStr ys -mvPunct (Space : x : ys) | isNote x = x : ys -mvPunct xs = xs - --- A replacement for citeproc-hs's endWithPunct, which wrongly treats --- a sentence ending in '.)' as not ending with punctuation, leading --- to an extra period. -endWithPunct :: [Inline] -> Bool -endWithPunct [] = True -endWithPunct xs@(_:_) = case reverse (stringify [last xs]) of - [] -> True - (')':c:_) | isEndPunct c -> True - (c:_) | isEndPunct c -> True - | otherwise -> False - where isEndPunct c = c `elem` ".,;:!?" - -deNote :: Pandoc -> Pandoc -deNote = topDown go - where go (Cite (c:cs) [Note xs]) = - Cite (c:cs) [Note $ bottomUp go' $ sanitize c xs] - go (Note xs) = Note $ bottomUp go' xs - go x = x - go' (Note [Para xs]:ys) = - if startWithPunct ys && endWithPunct xs - then initInline xs ++ ys - else xs ++ ys - go' xs = xs - sanitize :: Citation -> [Block] -> [Block] - sanitize Citation{citationPrefix = pref} [Para xs] = - case (null pref, endWithPunct xs) of - (True, False) -> [Para $ xs ++ [Str "."]] - (True, True) -> [Para xs] - (False, False) -> [Para $ toCapital $ xs ++ [Str "."]] - (False, True) -> [Para $ toCapital xs] - sanitize _ bs = bs - -isTextualCitation :: [Citation] -> Bool -isTextualCitation (c:_) = citationMode c == AuthorInText -isTextualCitation _ = False - --- | Retrieve all citations from a 'Pandoc' docuument. To be used with --- 'query'. -getCitation :: Inline -> [[Citation]] -getCitation i | Cite t _ <- i = [t] - | otherwise = [] - -setHash :: Citation -> State Int Citation -setHash c = do - ident <- get - put $ ident + 1 - return c{ citationHash = ident } - -toCslCite :: Citation -> CSL.Cite -toCslCite c - = let (l, s) = locatorWords $ citationSuffix c - (la,lo) = parseLocator l - s' = case (l,s) of - -- treat a bare locator as if it begins with space - -- so @item1 [blah] is like [@item1, blah] - ("",(x:_)) - | not (isPunct x) -> [Space] ++ s - _ -> s - isPunct (Str (x:_)) = isPunctuation x - isPunct _ = False - citMode = case citationMode c of - AuthorInText -> (True, False) - SuppressAuthor -> (False,True ) - NormalCitation -> (False,False) - in emptyCite { CSL.citeId = citationId c - , CSL.citePrefix = PandocText $ citationPrefix c - , CSL.citeSuffix = PandocText s' - , CSL.citeLabel = la - , CSL.citeLocator = lo - , CSL.citeNoteNumber = show $ citationNoteNum c - , CSL.authorInText = fst citMode - , CSL.suppressAuthor = snd citMode - , CSL.citeHash = citationHash c - } - -locatorWords :: [Inline] -> (String, [Inline]) -locatorWords inp = - case parse pLocatorWords "suffix" $ breakup inp of - Right r -> r - Left _ -> ("",inp) - where breakup [] = [] - breakup (Str x : xs) = map Str (splitup x) ++ breakup xs - breakup (x : xs) = x : breakup xs - splitup = groupBy (\x y -> x /= '\160' && y /= '\160') - -pLocatorWords :: Parsec [Inline] st (String, [Inline]) -pLocatorWords = do - l <- pLocator - s <- getInput -- rest is suffix - if length l > 0 && last l == ',' - then return (init l, Str "," : s) - else return (l, s) - -pMatch :: (Inline -> Bool) -> Parsec [Inline] st Inline -pMatch condition = try $ do - t <- anyToken - guard $ condition t - return t - -pSpace :: Parsec [Inline] st Inline -pSpace = pMatch (\t -> t == Space || t == Str "\160") - -pLocator :: Parsec [Inline] st String -pLocator = try $ do - optional $ pMatch (== Str ",") - optional pSpace - f <- (guardFollowingDigit >> return [Str "p"]) -- "page" the default - <|> many1 (notFollowedBy pSpace >> anyToken) - gs <- many1 pWordWithDigits - return $ stringify f ++ (' ' : unwords gs) - -guardFollowingDigit :: Parsec [Inline] st () -guardFollowingDigit = do - t <- lookAhead anyToken - case t of - Str (d:_) | isDigit d -> return () - _ -> mzero - -pWordWithDigits :: Parsec [Inline] st String -pWordWithDigits = try $ do - optional pSpace - r <- many1 (notFollowedBy pSpace >> anyToken) - let s = stringify r - guard $ any isDigit s - return s - diff --git a/src/Text/Pandoc/Options.hs b/src/Text/Pandoc/Options.hs index c7c37d6b8..48e418ab2 100644 --- a/src/Text/Pandoc/Options.hs +++ b/src/Text/Pandoc/Options.hs @@ -48,7 +48,6 @@ import Data.Set (Set) import qualified Data.Set as Set import Data.Default import Text.Pandoc.Highlighting (Style, pygments) -import qualified Text.CSL as CSL -- | Individually selectable syntax extensions. data Extension = @@ -205,8 +204,6 @@ data ReaderOptions = ReaderOptions{ , readerOldDashes :: Bool -- ^ Use pandoc <= 1.8.2.1 behavior -- in parsing dashes; -- is em-dash; -- - before numerial is en-dash - , readerReferences :: [CSL.Reference] -- ^ Bibliographic references - , readerCitationStyle :: Maybe CSL.Style -- ^ Citation style , readerApplyMacros :: Bool -- ^ Apply macros to TeX math , readerIndentedCodeClasses :: [String] -- ^ Default classes for -- indented code blocks @@ -223,8 +220,6 @@ instance Default ReaderOptions , readerColumns = 80 , readerTabStop = 4 , readerOldDashes = False - , readerReferences = [] - , readerCitationStyle = Nothing , readerApplyMacros = True , readerIndentedCodeClasses = [] , readerDefaultImageExtension = "" @@ -289,7 +284,6 @@ data WriterOptions = WriterOptions , writerSourceURL :: Maybe String -- ^ Absolute URL + directory of 1st source file , writerUserDataDir :: Maybe FilePath -- ^ Path of user data directory , writerCiteMethod :: CiteMethod -- ^ How to print cites - , writerBiblioFiles :: [FilePath] -- ^ Biblio files to use for citations , writerHtml5 :: Bool -- ^ Produce HTML5 , writerHtmlQTags :: Bool -- ^ Use @@ tags for quotes in HTML , writerBeamer :: Bool -- ^ Produce beamer LaTeX slide show @@ -332,7 +326,6 @@ instance Default WriterOptions where , writerSourceURL = Nothing , writerUserDataDir = Nothing , writerCiteMethod = Citeproc - , writerBiblioFiles = [] , writerHtml5 = False , writerHtmlQTags = False , writerBeamer = False diff --git a/src/Text/Pandoc/Readers/LaTeX.hs b/src/Text/Pandoc/Readers/LaTeX.hs index ded57df5a..e558ed1b9 100644 --- a/src/Text/Pandoc/Readers/LaTeX.hs +++ b/src/Text/Pandoc/Readers/LaTeX.hs @@ -38,7 +38,6 @@ import Text.Pandoc.Definition import Text.Pandoc.Walk import Text.Pandoc.Shared import Text.Pandoc.Options -import Text.Pandoc.Biblio (processBiblio) import Text.Pandoc.Parsing hiding ((<|>), many, optional, space) import qualified Text.Pandoc.UTF8 as UTF8 import Data.Char ( chr, ord ) @@ -67,9 +66,7 @@ parseLaTeX = do eof st <- getState let meta = stateMeta st - refs <- getOption readerReferences - mbsty <- getOption readerCitationStyle - let (Pandoc _ bs') = processBiblio mbsty refs $ doc bs + let (Pandoc _ bs') = doc bs return $ Pandoc meta bs' type LP = Parser [Char] ParserState diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs index 05662d9b5..658335202 100644 --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -54,7 +54,6 @@ import Text.Pandoc.Parsing hiding (tableWith) import Text.Pandoc.Readers.LaTeX ( rawLaTeXInline, rawLaTeXBlock ) import Text.Pandoc.Readers.HTML ( htmlTag, htmlInBalanced, isInlineTag, isBlockTag, isTextTag, isCommentTag ) -import Text.Pandoc.Biblio (processBiblio) import Data.Monoid (mconcat, mempty) import Control.Applicative ((<$>), (<*), (*>), (<$)) import Control.Monad @@ -327,9 +326,7 @@ parseMarkdown = do st <- getState let meta = runF (stateMeta' st) st let Pandoc _ bs = B.doc $ runF blocks st - mbsty <- getOption readerCitationStyle - refs <- getOption readerReferences - return $ processBiblio mbsty refs $ Pandoc meta bs + return $ Pandoc meta bs addWarning :: Maybe SourcePos -> String -> MarkdownParser () addWarning mbpos msg = diff --git a/src/Text/Pandoc/Writers/LaTeX.hs b/src/Text/Pandoc/Writers/LaTeX.hs index ab579a326..6a781ddec 100644 --- a/src/Text/Pandoc/Writers/LaTeX.hs +++ b/src/Text/Pandoc/Writers/LaTeX.hs @@ -43,7 +43,6 @@ import Data.Char ( toLower, isPunctuation ) import Control.Applicative ((<|>)) import Control.Monad.State import Text.Pandoc.Pretty -import System.FilePath (dropExtension) import Text.Pandoc.Slides import Text.Pandoc.Highlighting (highlight, styleToLaTeX, formatLaTeXInline, formatLaTeXBlock, @@ -120,7 +119,6 @@ pandocToLaTeX options (Pandoc meta blocks) = do (biblioTitle :: String) <- liftM (render colwidth) $ inlineListToLaTeX lastHeader let main = render colwidth $ vsep body st <- get - let biblioFiles = intercalate "," $ map dropExtension $ writerBiblioFiles options let context = defField "toc" (writerTableOfContents options) $ defField "toc-depth" (show (writerTOCDepth options - if writerChapters options @@ -152,11 +150,9 @@ pandocToLaTeX options (Pandoc meta blocks) = do $ writerHighlightStyle options ) else id) $ (case writerCiteMethod options of - Natbib -> defField "biblio-files" biblioFiles . - defField "biblio-title" biblioTitle . + Natbib -> defField "biblio-title" biblioTitle . defField "natbib" True - Biblatex -> defField "biblio-files" biblioFiles . - defField "biblio-title" biblioTitle . + Biblatex -> defField "biblio-title" biblioTitle . defField "biblatex" True _ -> id) $ metadata diff --git a/tests/Tests/Old.hs b/tests/Tests/Old.hs index 8609781d0..5054559a1 100644 --- a/tests/Tests/Old.hs +++ b/tests/Tests/Old.hs @@ -63,7 +63,6 @@ tests = [ testGroup "markdown" "markdown-reader-more.txt" "markdown-reader-more.native" , lhsReaderTest "markdown+lhs" ] - , testGroup "citations" markdownCitationTests ] , testGroup "rst" [ testGroup "writer" (writerTests "rst" ++ lhsWriterTests "rst") @@ -190,19 +189,6 @@ fb2WriterTest title opts inputfile normfile = ignoreBinary = unlines . filter (not . startsWith " [String] -- ^ Options to pass to pandoc -- cgit v1.2.3 From 8e61a6214ffda2f76980dbb36542c1d36951150c Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 1 Sep 2013 09:45:29 -0700 Subject: Document fact that --toc doesn't do anything for docx or odt output. See #458. --- README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index f85e62e14..7b75565c1 100644 --- a/README +++ b/README @@ -342,7 +342,7 @@ General writer options : Include an automatically generated table of contents (or, in the case of `latex`, `context`, and `rst`, an instruction to create one) in the output document. This option has no effect on `man`, - `docbook`, `slidy`, `slideous`, or `s5` output. + `docbook`, `slidy`, `slideous`, `s5`, `docx`, or `odt` output. `--toc-depth=`*NUMBER* : Specify the number of section levels to include in the table -- cgit v1.2.3 From 8977b2aaed096a71e669e1161f6ce10d5286f34e Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 1 Sep 2013 16:22:40 -0700 Subject: Changed --metadata to return Boolean True if no value. Also documented in README. --- README | 6 ++++++ pandoc.hs | 15 ++++++++------- 2 files changed, 14 insertions(+), 7 deletions(-) (limited to 'README') diff --git a/README b/README index 7b75565c1..236637d10 100644 --- a/README +++ b/README @@ -284,6 +284,12 @@ Reader options if no directory is provided. If you want to run a script in the working directory, preface the filename with `./`. +`-M` *KEY[=VAL]*, `--metadata=`*KEY[:VAL]* +: Set the metadata field *KEY* to the value *VAL* after + parsing. A value specified on the command line overrides a value + specified in the document. Values will be interpreted as raw strings. + If no value is specified, the value will be treated as Boolean true. + `--normalize` : Normalize the document after reading: merge adjacent `Str` or `Emph` elements, for example, and remove repeated `Space`s. diff --git a/pandoc.hs b/pandoc.hs index 3713467ad..0bc2d7359 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -113,7 +113,7 @@ data Opt = Opt , optTransforms :: [Pandoc -> Pandoc] -- ^ Doc transforms to apply , optTemplate :: Maybe FilePath -- ^ Custom template , optVariables :: [(String,String)] -- ^ Template variables to set - , optMetadata :: [(String,String)] -- ^ Metadata fields to set + , optMetadata :: [(String,MetaValue)] -- ^ Metadata fields to set , optOutputFile :: String -- ^ Name of output file , optNumberSections :: Bool -- ^ Number sections in LaTeX , optNumberOffset :: [Int] -- ^ Starting number for sections @@ -328,8 +328,8 @@ options = (ReqArg (\arg opt -> do let (key,val) = case break (`elem` ":=") arg of - (k,_:v) -> (k,v) - (k,_) -> (k,"true") + (k,_:v) -> (k, MetaString v) + (k,_) -> (k, MetaBool True) return opt{ optMetadata = (key,val) : optMetadata opt }) "KEY[:VALUE]") "" @@ -658,7 +658,7 @@ options = , Option "" ["bibliography"] (ReqArg (\arg opt -> - return opt{ optMetadata = ("bibliography",arg) : + return opt{ optMetadata = ("bibliography",MetaString arg) : optMetadata opt , optPlugins = externalFilter "pandoc-citeproc" : optPlugins opt @@ -669,7 +669,7 @@ options = , Option "" ["csl"] (ReqArg (\arg opt -> - return opt{ optMetadata = ("csl",arg) : + return opt{ optMetadata = ("csl", MetaString arg) : optMetadata opt }) "FILE") "" @@ -677,7 +677,8 @@ options = , Option "" ["citation-abbreviations"] (ReqArg (\arg opt -> - return opt{ optMetadata = ("citation-abbreviations",arg) : + return opt{ optMetadata = ("citation-abbreviations", + MetaString arg) : optMetadata opt }) "FILE") "" @@ -1104,7 +1105,7 @@ main = do reader readerOpts - let doc0 = foldr (\(k,v) -> setMeta k (MetaString v)) doc metadata + let doc0 = foldr (\(k,v) -> setMeta k v) doc metadata let doc1 = foldr ($) doc0 transforms doc2 <- foldrM ($) doc1 $ map ($ [writerName']) plugins -- cgit v1.2.3 From 9ca89ec673b3f531927793799f8dfecc9fbf85c1 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 1 Sep 2013 16:26:08 -0700 Subject: Documented --bibliography, --csl, --citation-abbreviations. --- README | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) (limited to 'README') diff --git a/README b/README index 236637d10..dc08c374d 100644 --- a/README +++ b/README @@ -604,6 +604,23 @@ Options affecting specific writers Citation rendering ------------------ +`--bibliography=`*FILE* +: Set the `bibliography` field in the document's metadata to *FILE*, + overriding any value set in the metadata, and tell pandoc to + use the `pandoc-citeproc` filter. (This is equivalent to the + combination `--metadata bibliography=FILE --filter pandoc-citeproc`.) + Note that an error will result unless `pandoc-citeproc` is installed. + +`--csl=`*FILE* +: Set the `csl` field in the document's metadata to *FILE*, + overriding any value set in the metadata. (This is equivalent to + `--metadata csl=FILE`.) + +`--citation-abbreviations=`*FILE* +: Set the `citation-abbreviations` field in the document's metadata to + *FILE*, overriding any value set in the metadata. (This is equivalent to + `--metadata citation-abbreviations=FILE`.) + `--natbib` : Use natbib for citations in LaTeX output. -- cgit v1.2.3 From 6fb9e82281250a4d4b8ddf3eb38c4974b321e5a5 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Thu, 5 Sep 2013 19:55:19 -0700 Subject: Fixed typo. --- README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index dc08c374d..0a04fb6ab 100644 --- a/README +++ b/README @@ -277,7 +277,7 @@ Reader options Filters may be written in any language. `Text.Pandoc.JSON` exports `toJSONFilter` to facilitate writing filters in Haskell. - Those who would prefer to write filters in pandoc can use the + Those who would prefer to write filters in python can use the module `pandoc.py`: see for the module and several examples. Note that the *EXECUTABLE* will be sought in the user's `PATH`, and not in the working directory, -- cgit v1.2.3 From 5afd373ae45f525ff1eff5e54c1850fe2c614b4b Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sat, 7 Sep 2013 09:36:37 -0700 Subject: Added `lists_without_preceding_blankline` extension. * Added `Ext_lists_without_preceding_blankline` to `Extension` in `Options`. Added this option to `githubMarkdownExtensions`. * Made markdown reader sensitive to this. * Closes #972. --- README | 4 ++++ src/Text/Pandoc/Options.hs | 2 ++ src/Text/Pandoc/Readers/Markdown.hs | 1 + 3 files changed, 7 insertions(+) (limited to 'README') diff --git a/README b/README index 0a04fb6ab..7d926216b 100644 --- a/README +++ b/README @@ -2454,6 +2454,10 @@ in pandoc, but may be enabled by adding `+EXTENSION` to the format name, where `EXTENSION` is the name of the extension. Thus, for example, `markdown+hard_line_breaks` is markdown with hard line breaks. +**Extension: `lists_without_preceding_blankline`**\ +Allow a list to occur right after a paragraph, with no intervening +blank space. + **Extension: `hard_line_breaks`**\ Causes all newlines within a paragraph to be interpreted as hard line breaks instead of spaces. diff --git a/src/Text/Pandoc/Options.hs b/src/Text/Pandoc/Options.hs index 48e418ab2..5f65abdde 100644 --- a/src/Text/Pandoc/Options.hs +++ b/src/Text/Pandoc/Options.hs @@ -80,6 +80,7 @@ data Extension = | Ext_link_attributes -- ^ MMD style reference link attributes | Ext_autolink_bare_uris -- ^ Make all absolute URIs into links | Ext_fancy_lists -- ^ Enable fancy list numbers and delimiters + | Ext_lists_without_preceding_blankline -- ^ Allow lists without preceding blank | Ext_startnum -- ^ Make start number of ordered list significant | Ext_definition_lists -- ^ Definition lists as in pandoc, mmd, php | Ext_example_lists -- ^ Markdown-style numbered examples @@ -169,6 +170,7 @@ githubMarkdownExtensions = Set.fromList , Ext_intraword_underscores , Ext_strikeout , Ext_hard_line_breaks + , Ext_lists_without_preceding_blankline ] multimarkdownExtensions :: Set Extension diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs index 9b98cbc3e..2ca0d312a 100644 --- a/src/Text/Pandoc/Readers/Markdown.hs +++ b/src/Text/Pandoc/Readers/Markdown.hs @@ -1559,6 +1559,7 @@ endline :: MarkdownParser (F Inlines) endline = try $ do newline notFollowedBy blankline + guardDisabled Ext_lists_without_preceding_blankline <|> notFollowedBy listStart guardEnabled Ext_blank_before_blockquote <|> notFollowedBy emailBlockQuoteStart guardEnabled Ext_blank_before_header <|> notFollowedBy (char '#') -- atx header -- parse potential list-starts differently if in a list: -- cgit v1.2.3 From 81e2df32c92ee95771f2613b9ad30aeaf11423e5 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 8 Sep 2013 15:47:50 -0700 Subject: Made . . . for pause work in all slide show formats except slideous. --- README | 5 ++--- changelog | 5 +---- src/Text/Pandoc/Writers/HTML.hs | 22 +++++++++++++++------- 3 files changed, 18 insertions(+), 14 deletions(-) (limited to 'README') diff --git a/README b/README index 7d926216b..56ad50b3c 100644 --- a/README +++ b/README @@ -2680,9 +2680,8 @@ a single document. Inserting pauses ---------------- -In reveal.js and beamer slide shows, you can add "pauses" within -a slide by including a paragraph containing three dots, separated -by spaces: +You can add "pauses" within a slide by including a paragraph containing +three dots, separated by spaces: # Slide with a pause diff --git a/changelog b/changelog index b7091460f..b3e9d8b0b 100644 --- a/changelog +++ b/changelog @@ -413,7 +413,7 @@ as markdown citations, it is redundant to have a bibliography, since one will be generated automatically.) - * Added syntax for "pauses" in beamer or reaveljs slide shows. + * Added syntax for "pauses" in slide shows: This gives @@ -421,9 +421,6 @@ a pause. - [note - no longer seems to work in recente revealjs - perhaps - this should be reverted] - * Use new flexible metadata type. + Depend on `pandoc-types` 1.12. This changes the type of diff --git a/src/Text/Pandoc/Writers/HTML.hs b/src/Text/Pandoc/Writers/HTML.hs index 63b466af3..78a3edce8 100644 --- a/src/Text/Pandoc/Writers/HTML.hs +++ b/src/Text/Pandoc/Writers/HTML.hs @@ -268,11 +268,24 @@ elementToHtml slideLevel opts (Sec level num (id',classes,keyvals) title' elemen else blockToHtml opts (Header level' (id',classes,keyvals) title') let isSec (Sec _ _ _ _ _) = True isSec (Blk _) = False + let isPause (Blk x) = x == Para [Str ".",Space,Str ".",Space,Str "."] + isPause _ = False + let fragmentClass = case writerSlideVariant opts of + RevealJsSlides -> "fragment" + _ -> "incremental" + let inDiv xs = Blk (RawBlock (Format "html") ("
")) : + (xs ++ [Blk (RawBlock (Format "html") "
")]) innerContents <- mapM (elementToHtml slideLevel opts) $ if titleSlide -- title slides have no content of their own then filter isSec elements - else elements + else if slide + then case splitBy isPause elements of + [] -> [] + [x] -> x + xs -> concatMap inDiv xs + else elements let inNl x = mconcat $ nl opts : intersperse (nl opts) x ++ [nl opts] let classes' = ["titleslide" | titleSlide] ++ ["slide" | slide] ++ ["section" | (slide || writerSectionDivs opts) && @@ -401,10 +414,6 @@ blockToHtml opts (Para [Image txt (s,'f':'i':'g':':':tit)]) = do [nl opts, img, capt, nl opts] else H.div ! A.class_ "figure" $ mconcat [nl opts, img, capt, nl opts] --- . . . indicates a pause in a slideshow -blockToHtml opts (Para [Str ".",Space,Str ".",Space,Str "."]) - | writerSlideVariant opts == RevealJsSlides = - blockToHtml opts (RawBlock "html" "
") blockToHtml opts (Para lst) = do contents <- inlineListToHtml opts lst return $ H.p contents @@ -580,8 +589,7 @@ toListItem opts item = nl opts >> H.li item blockListToHtml :: WriterOptions -> [Block] -> State WriterState Html blockListToHtml opts lst = - mapM (blockToHtml opts) lst >>= - return . mconcat . intersperse (nl opts) + fmap (mconcat . intersperse (nl opts)) $ mapM (blockToHtml opts) lst -- | Convert list of Pandoc inline elements to HTML. inlineListToHtml :: WriterOptions -> [Inline] -> State WriterState Html -- cgit v1.2.3 From 4381c37b100a4cfd14020458e6b4e340a02b851e Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 8 Sep 2013 22:45:11 -0700 Subject: `--bibliography` no longer implies `-F pandoc-citeproc`. This could lead to double filtering if the user specifies `-F` too. --- README | 6 ++---- pandoc.hs | 2 -- 2 files changed, 2 insertions(+), 6 deletions(-) (limited to 'README') diff --git a/README b/README index 56ad50b3c..7a827ef7b 100644 --- a/README +++ b/README @@ -606,10 +606,8 @@ Citation rendering `--bibliography=`*FILE* : Set the `bibliography` field in the document's metadata to *FILE*, - overriding any value set in the metadata, and tell pandoc to - use the `pandoc-citeproc` filter. (This is equivalent to the - combination `--metadata bibliography=FILE --filter pandoc-citeproc`.) - Note that an error will result unless `pandoc-citeproc` is installed. + overriding any value set in the metadata. (This is equivalent to + `--metadata bibliography=FILE`.) `--csl=`*FILE* : Set the `csl` field in the document's metadata to *FILE*, diff --git a/pandoc.hs b/pandoc.hs index 0bc2d7359..2c20e16b5 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -660,8 +660,6 @@ options = (\arg opt -> return opt{ optMetadata = ("bibliography",MetaString arg) : optMetadata opt - , optPlugins = externalFilter "pandoc-citeproc" - : optPlugins opt }) "FILE") "" -- cgit v1.2.3 From d47eadaf5645a807385f62b5b265274594182967 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Tue, 10 Sep 2013 20:23:03 -0700 Subject: `--bibliography` again implies `-F pandoc-citeproc`. But only if pandoc-citeproc isn't already specified as a filter. --- README | 5 +++-- pandoc.hs | 16 +++++++++++----- 2 files changed, 14 insertions(+), 7 deletions(-) (limited to 'README') diff --git a/README b/README index 7a827ef7b..ddc2697be 100644 --- a/README +++ b/README @@ -606,8 +606,9 @@ Citation rendering `--bibliography=`*FILE* : Set the `bibliography` field in the document's metadata to *FILE*, - overriding any value set in the metadata. (This is equivalent to - `--metadata bibliography=FILE`.) + overriding any value set in the metadata, and process citations + using `pandoc-citeproc`. (This is equivalent to + `--metadata bibliography=FILE --filter pandoc-citeproc`.) `--csl=`*FILE* : Set the `csl` field in the document's metadata to *FILE*, diff --git a/pandoc.hs b/pandoc.hs index 2c20e16b5..7507e8334 100644 --- a/pandoc.hs +++ b/pandoc.hs @@ -140,7 +140,7 @@ data Opt = Opt , optReferenceLinks :: Bool -- ^ Use reference links in writing markdown, rst , optWrapText :: Bool -- ^ Wrap text , optColumns :: Int -- ^ Line length in characters - , optPlugins :: [[String] -> Pandoc -> IO Pandoc] -- ^ Plugins to apply + , optFilters :: [FilePath] -- ^ Filters to apply , optEmailObfuscation :: ObfuscationMethod , optIdentifierPrefix :: String , optIndentedCodeClasses :: [String] -- ^ Default classes for indented code blocks @@ -195,7 +195,7 @@ defaultOpts = Opt , optReferenceLinks = False , optWrapText = True , optColumns = 72 - , optPlugins = [] + , optFilters = [] , optEmailObfuscation = JavascriptObfuscation , optIdentifierPrefix = "" , optIndentedCodeClasses = [] @@ -285,8 +285,7 @@ options = , Option "F" ["filter"] (ReqArg - (\arg opt -> return opt { optPlugins = externalFilter arg : - optPlugins opt }) + (\arg opt -> return opt { optFilters = arg : optFilters opt }) "PROGRAM") "" -- "External JSON filter" @@ -913,7 +912,7 @@ main = do , optReferenceLinks = referenceLinks , optWrapText = wrap , optColumns = columns - , optPlugins = plugins + , optFilters = filters , optEmailObfuscation = obfuscationMethod , optIdentifierPrefix = idPrefix , optIndentedCodeClasses = codeBlockClasses @@ -933,6 +932,13 @@ main = do mapM_ (\arg -> UTF8.hPutStrLn stdout arg) args exitWith ExitSuccess + -- --bibliography implies -F pandoc-citeproc for backwards compatibility: + let filters' = case lookup "bibliography" metadata of + Just _ | "pandoc-citeproc" `notElem` filters -> + "pandoc-citeproc" : filters + _ -> filters + let plugins = map externalFilter filters' + let sources = if ignoreArgs then [] else args datadir <- case mbDataDir of -- cgit v1.2.3 From 6a5d7cfcbee3c57a27a8d3ec1d53438d2914090a Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Wed, 11 Sep 2013 19:43:13 -0700 Subject: Changed pandocfilters repository URL. --- README | 2 +- changelog | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) (limited to 'README') diff --git a/README b/README index ddc2697be..0a9d8593f 100644 --- a/README +++ b/README @@ -278,7 +278,7 @@ Reader options Filters may be written in any language. `Text.Pandoc.JSON` exports `toJSONFilter` to facilitate writing filters in Haskell. Those who would prefer to write filters in python can use the - module `pandoc.py`: see + module `pandoc.py`: see for the module and several examples. Note that the *EXECUTABLE* will be sought in the user's `PATH`, and not in the working directory, if no directory is provided. If you want to run a script in the diff --git a/changelog b/changelog index 6a6d258a4..f2742664c 100644 --- a/changelog +++ b/changelog @@ -44,7 +44,7 @@ (Pandoc AST transformations that operate on JSON serializations). Filters are always passed the name of the output format, so their behavior can be tailored to it. The repository - contains + contains a python module for writing pandoc filters in python, with a number of examples. -- cgit v1.2.3 From 873a21170e1749366f0f6844c884ccf3a40e2c92 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Wed, 11 Sep 2013 22:17:00 -0700 Subject: Updated README on pandocfilters python module. --- README | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) (limited to 'README') diff --git a/README b/README index 0a9d8593f..95dd4797b 100644 --- a/README +++ b/README @@ -278,11 +278,12 @@ Reader options Filters may be written in any language. `Text.Pandoc.JSON` exports `toJSONFilter` to facilitate writing filters in Haskell. Those who would prefer to write filters in python can use the - module `pandoc.py`: see - for the module and several examples. Note that the *EXECUTABLE* - will be sought in the user's `PATH`, and not in the working directory, - if no directory is provided. If you want to run a script in the - working directory, preface the filename with `./`. + module `pandocfilters`, installable from PyPI. See + for the module and several + examples. Note that the *EXECUTABLE* will be sought in the user's + `PATH`, and not in the working directory, if no directory is + provided. If you want to run a script in the working directory, + preface the filename with `./`. `-M` *KEY[=VAL]*, `--metadata=`*KEY[:VAL]* : Set the metadata field *KEY* to the value *VAL* after -- cgit v1.2.3 From c92d9db7f096a1c5102f5c2a1e506e980886859d Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Fri, 13 Sep 2013 22:40:00 -0700 Subject: README: Added link to lua. --- README | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index 95dd4797b..0cc37e655 100644 --- a/README +++ b/README @@ -2837,5 +2837,5 @@ Jamie F. Olson. [PDF]: http://www.adobe.com/pdf/ [reveal.js]: http://lab.hakim.se/reveal-js/ [FictionBook2]: http://www.fictionbook.org/index.php/Eng:XML_Schema_Fictionbook_2.1 -[lua]: TODO +[lua]: http://www.lua.org -- cgit v1.2.3 From 87def47e87d7d8ba20dabc323e7b91a424b95406 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Thu, 19 Sep 2013 21:01:17 -0700 Subject: Documented `--metadata` change. --- README | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) (limited to 'README') diff --git a/README b/README index 0cc37e655..6c9c9d3ed 100644 --- a/README +++ b/README @@ -286,10 +286,10 @@ Reader options preface the filename with `./`. `-M` *KEY[=VAL]*, `--metadata=`*KEY[:VAL]* -: Set the metadata field *KEY* to the value *VAL* after - parsing. A value specified on the command line overrides a value - specified in the document. Values will be interpreted as raw strings. - If no value is specified, the value will be treated as Boolean true. +: Set the metadata field *KEY* to the value *VAL*. A value specified + on the command line overrides a value specified in the document. + Values will be parsed as YAML boolean or string values. If no value is + specified, the value will be treated as Boolean true. `--normalize` : Normalize the document after reading: merge adjacent -- cgit v1.2.3 From df313b859f19857a9773627d60bf2e6c8d6ba9b3 Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Thu, 19 Sep 2013 21:04:58 -0700 Subject: Added small explanation of difference btw `-M` and `-V`. --- README | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) (limited to 'README') diff --git a/README b/README index 6c9c9d3ed..ba1f93661 100644 --- a/README +++ b/README @@ -289,7 +289,11 @@ Reader options : Set the metadata field *KEY* to the value *VAL*. A value specified on the command line overrides a value specified in the document. Values will be parsed as YAML boolean or string values. If no value is - specified, the value will be treated as Boolean true. + specified, the value will be treated as Boolean true. Like + `--variable`, `--metadata` causes template variables to be set. + But unlike `--variable`, `--metadata` affects the metadata of the + underlying document (which is accessible from filters and may be + printed in some output formats). `--normalize` : Normalize the document after reading: merge adjacent -- cgit v1.2.3 From 0df7cce37da162c656aa88ecb67788109749668c Mon Sep 17 00:00:00 2001 From: John MacFarlane Date: Sun, 13 Oct 2013 15:36:19 -0700 Subject: Treat div with class "notes" as speaker notes in slide formats. Currently beamer goes to `\note{}`, revealjs to `