diff options
| -rw-r--r-- | changelog.md | 280 |
1 files changed, 182 insertions, 98 deletions
diff --git a/changelog.md b/changelog.md index b13bf1fd4..a48b668b5 100644 --- a/changelog.md +++ b/changelog.md @@ -2,6 +2,11 @@ ## pandoc 2.12 (UNRELEASED -- PROVISIONAL) + * When downloading content from URL arguments, be sensitive to + the character encoding (#5600). We can properly handle UTF-8 and latin1 + (ISO-8859-1); for others we raise an error. Fall back to latin1 if + no charset is given in the mime type and UTF-8 decoding fails. + * Add new unexported module Text.Pandoc.XML.Light, as well as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc, Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091). @@ -34,75 +39,6 @@ these have been added. And the docx golden tests have been updated, because the new parser does not preserve the order of attributes. - * Text.Pandoc.App - - + Add `parseOptionsFromArgs` [API change, new exported function]. - - * Text.Pandoc.Citeproc.BibTeX - - + `Text.Pandoc.Citeproc.writeBibTeXString` now returns - `Doc Text` instead of `Text` (#7068). - + Correctly handle `pages` (= `page` in CSL) (#7067). - + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067). - + In BibTeX output, protect foreign titles since there's no language - field (#7067). - + Clean up BibTeX parsing (#7049). Previously there was a messy code - path that gave strange results in some cases, not passing through raw - tex but trying to extract a string content. This was an artefact of - trying to handle some special bibtex-specific commands in the BibTeX - reader. Now we just handle these in the LaTeX reader and simplify - parsing in the BibTeX reader. This does mean that more raw tex will - be passed through (and currently this is not sensitive to the - `raw_tex` extension; this should be fixed). - - * Text.Pandoc.Citeproc.MetaValue - - + Correctly parse "raw" date value in markdown references metadata. - (See jgm/citeproc#53.) - - * Text.Pandoc.Class - - + Add `getTimestamp` [API change]. This attempts to read the - `SOURCE_DATE_EPOCH` environment variable and parse a UTC time - from it (treating it as a unix date stamp, see - https://reproducible-builds.org/specs/source-date-epoch/). If the - variable is not set or can't be parsed as a unix date stamp, then the - function returns the current date. - - * Text.Pandoc.Error - - + Remove unused variables (Albert Krewinkel) - + Export `renderError` [API change]. - + Refactor `handleError` to use `renderError`. This allows us render - error messages without exiting. - - * Text.Pandoc.Extensions - - + `Ext_task_lists` is now supported by org (and turned - on by default) (Albert Krewinkel, #6336). - + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes - (#7097). This attribute was listed as allowed, but it didn't actually - do anything. Use `attributes` for code attributes and more. - - * Lua subsystem: - - + Always load built-in Lua scripts from default data-dir (Albert - Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always - loaded from the system's default data directory. Loading from a - different directory by overriding the default path, e.g. via - `--data-dir`, is no longer supported to avoid unexpected behavior - and to address security concerns. - + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565). - The module allows to work with file paths in a convenient and - platform-independent manner. - - * Text.Pandoc.PDF - - + Disable `smart` extension when building PDF via LaTeX. - This is to prevent accidental creation of ligatures like - `` ?` `` and `` !` `` (especially in languages with quotations like - German), and similar ligature issues. (See jgm/citeproc#54.) - * DocBook reader: + Avoid expensive tree normalization step, as it is not necessary @@ -137,6 +73,11 @@ * LaTeX reader: + + Don't export `tokenize`, `untokenize` [API change]. These are internal + implementation details, which were only exported for testing. + They don't belong in the public API. + + Improved efficiency of the parser. With these changes the reader + is almost twice as fast as in the last release in our benchmarks. + Code cleanup, removing some unnecessary things. + Rewrite `withRaw` so it doesn't rely on fragile assumptions about token positions (which break when macros are expanded) @@ -156,11 +97,19 @@ it doesn't change anything. + Improve `braced'`. Remove the parameter, have it parse the opening brace, and make it more efficient. + + Factor out pieces of the LaTeX reader to make the module smaller. + This reduces memory demands when compiling. Created + Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Accent}. + Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map + instead of individual commands. + + * Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change]. * Markdown reader: + Improved handling of mmd link attributes in references (#7080). Previously they only worked for links that had titles. + + Improved efficiency of the parser (benchmarks show a 15% speedup). * OPML reader: @@ -218,7 +167,7 @@ + Use `getTimestamp` instead of `getCurrentTime` for timestamp. Setting `SOURCE_DATE_EPOCH` will allow reproducible builds. - * Text.Pandoc.Writers.EPUB + * EPUB writer: + Use `getTimestamp` instead of `getCurrentTime` for timestamp. Setting `SOURCE_DATE_EPOCH` will allow reproducible builds (#7093). @@ -233,6 +182,10 @@ contains an ampersand or another character with a special meaning in XML. + * Jira writer: + + + Use Span identifiers as anchors (Albert Krewinkel, + * LaTeX writer: + Adjust hypertargets to beginnings of paragraphs (#7078). @@ -241,6 +194,8 @@ This makes a particular difference for links to citations using `--citeproc` and `link-citations: true`. + Change BCP47 lang tag from `jp` to `ja` (Mauro Bieg, #7047). + + Use function instead of map for accent lookup (should be + more efficient). * Markdown writer: @@ -279,40 +234,151 @@ `--include-in-header`, `--include-before-body`, `--include-after-body`) may be used. - * LaTeX template: Update to iftex package (#7073) (Andrew Dunning) + * LaTeX template: + + + Update to iftex package (#7073) (Andrew Dunning) + + Wrap url colours in braces (#7121) (Loïc Grobol). * revealjs template: Add 'center' option for vertical slide centering. (maurerle, #7104). * Text.Pandoc.XML: Improve efficiency of `fromEntities`. - * Test suite: a more robust way of testing the executable. - Many of our tests require running the pandoc executable. This is - problematic for a few different reasons. First, cabal-install will - sometimes run the test suite after building the library but before - building the executable, which means the executable isn't in place for - the tests. One can work around that by first building, then building and - running the tests, but that's fragile. Second, we have to find the - executable. So far, we've done that using a function `findPandoc` that - attempts to locate it relative to the test executable (which can be - located using findExecutablePath). But the logic here is delicate and - work with every combination of options. To solve both problems, we add - an `--emulate` option to the `test-pandoc` executable. When `--emulate` - occurs as the first argument passed to `test-pandoc`, the program simply - emulates the regular pandoc executable, using the rest of the arguments - (after `--emulate`). Thus, `test-pandoc --emulate -f markdown -t latex` - is just like `pandoc -f markdown -t latex`. - Since all the work is done by library functions, implementing this - emulation just takes a couple lines of code and should be entirely - reliable. With this change, we can test the pandoc executable by running - the test program itself (locatable using `findExecutablePath`) with the - `--emulate` option. This removes the need for the fragile `findPandoc` - step, and it means we can run our integration tests even when we're just - building the library, not the executable. [Note: part of this change - involved simplifying some complex handling to set environment variables - for dynamic library paths. I have tested a build with - `--enable-dynamic-executable`, and it works, but further testing may be - needed.] + * Text.Pandoc.MIME + + + Add exported function `getCharset` [API change]. + + * Text.Pandoc.UTF8: change IO functions to return Text, not String + [API change]. This affects `readFile`, `getContents`, `writeFileWith`, + `writeFile`, `putStrWith`, `putStr`, `putStrLnWith`, `putStrLn`. + `hPutStrWith`, `hPutStr`, `hPutStrLnWith`, `hPutStrLn`, `hGetContents`. + This avoids the need to uselessly create a linked list of characters + when emiting output. + + * Text.Pandoc.App + + + Add `parseOptionsFromArgs` [API change, new exported function]. + + * Text.Pandoc.Citeproc.BibTeX + + + `Text.Pandoc.Citeproc.writeBibTeXString` now returns + `Doc Text` instead of `Text` (#7068). + + Correctly handle `pages` (= `page` in CSL) (#7067). + + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067). + + In BibTeX output, protect foreign titles since there's no language + field (#7067). + + Clean up BibTeX parsing (#7049). Previously there was a messy code + path that gave strange results in some cases, not passing through raw + tex but trying to extract a string content. This was an artefact of + trying to handle some special bibtex-specific commands in the BibTeX + reader. Now we just handle these in the LaTeX reader and simplify + parsing in the BibTeX reader. This does mean that more raw tex will + be passed through (and currently this is not sensitive to the + `raw_tex` extension; this should be fixed). + + * Text.Pandoc.Citeproc.MetaValue + + + Correctly parse "raw" date value in markdown references metadata. + (See jgm/citeproc#53.) + + * Text.Pandoc.Citeproc + + + Use https URLs for links (Salim B, #7122). + + * Text.Pandoc.Class + + + Add `getTimestamp` [API change]. This attempts to read the + `SOURCE_DATE_EPOCH` environment variable and parse a UTC time + from it (treating it as a unix date stamp, see + https://reproducible-builds.org/specs/source-date-epoch/). If the + variable is not set or can't be parsed as a unix date stamp, then the + function returns the current date. + + * Text.Pandoc.Error + + + Add `PandocUnsupportedCharsetError` constructor for + `PandocError` [API change]. + + Export `renderError` [API change]. + + Refactor `handleError` to use `renderError`. This allows us render + error messages without exiting. + + Remove unused variables (Albert Krewinkel) + + * Text.Pandoc.Extensions + + + `Ext_task_lists` is now supported by org (and turned + on by default) (Albert Krewinkel, #6336). + + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes + (#7097). This attribute was listed as allowed, but it didn't actually + do anything. Use `attributes` for code attributes and more. + + * Lua subsystem: + + + Always load built-in Lua scripts from default data-dir (Albert + Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always + loaded from the system's default data directory. Loading from a + different directory by overriding the default path, e.g. via + `--data-dir`, is no longer supported to avoid unexpected behavior + and to address security concerns. + + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565). + The module allows to work with file paths in a convenient and + platform-independent manner. + + Use strict evaluation when retrieving AST value from the stack + (Albert Krewinkel, #6674). + + * Text.Pandoc.PDF + + + Disable `smart` extension when building PDF via LaTeX. + This is to prevent accidental creation of ligatures like + `` ?` `` and `` !` `` (especially in languages with quotations like + German), and similar ligature issues. (See jgm/citeproc#54.) + + * Text.Pandoc.CSV: + + + Fix parsing of unquoted values (#7112). Previously we didn't allow + unescaped quotes in unquoted values, but they are allowed + in CSV. + + * Test suite: + + + Use a more robust method for testing the executable. Many + of our tests require running the pandoc executable. This + is problematic for a few different reasons. First, + cabal-install will sometimes run the test suite after + building the library but before building the executable, + which means the executable isn't in place for the tests. + One can work around that by first building, then building + and running the tests, but that's fragile. Second, we + have to find the executable. So far, we've done that using + a function `findPandoc` that attempts to locate it + relative to the test executable (which can be located + using findExecutablePath). But the logic here is delicate + and work with every combination of options. To solve both + problems, we add an `--emulate` option to the + `test-pandoc` executable. When `--emulate` occurs as the + first argument passed to `test-pandoc`, the program simply + emulates the regular pandoc executable, using the rest of + the arguments (after `--emulate`). Thus, `test-pandoc + --emulate -f markdown -t latex` is just like `pandoc -f + markdown -t latex`. Since all the work is done by library + functions, implementing this emulation just takes a couple + lines of code and should be entirely reliable. With this + change, we can test the pandoc executable by running the + test program itself (locatable using `findExecutablePath`) + with the `--emulate` option. This removes the need for the + fragile `findPandoc` step, and it means we can run our + integration tests even when we're just building the + library, not the executable. [Note: part of this change + involved simplifying some complex handling to set + environment variables for dynamic library paths. I have + tested a build with `--enable-dynamic-executable`, and it + works, but further testing may be needed.] + + Print accurate location if a test fails (Albert + Krewinkel). Ensures that tasty-hunit reports the location + of the failing test instead of the location of the helper + `test` function. + + * Documentation: Update URLs and use `https` where possible (#7122, + Salim B). * MANUAL.txt @@ -328,6 +394,9 @@ * Makefile: in `make bench`, create CSV files for comparison and compare against previous benchmark run. Add timestamp to CSV filenames. + * cabal.project: don't explicitly set -trypandoc. + If we do, this can't be overridden on the cabal command line. + * doc/lua-filters.md: improve documentation for `pandoc.mediabag.insert`, `pandoc.mediabag.fetch`, `directory`, `normalize` (Albert Krewinkel). @@ -339,6 +408,14 @@ * Require citeproc 0.3.0.7, which correctly titlecases when titles contain non-ASCII characters. + * Use skylighting 0.10.4. This version of skylighting uses xml-conduit + rather than hxt. This speeds up parsing of XML syntax definitions + fourfold, and removes four packages from pandoc's dependency graph: + hxt-charproperties, hxt-unicode, hxt-regex-xmlschema, hxt. + + * Add script `tools/parseTimings.pl` to help pin down which + modules take the most time and memory to compile. + * Avoid unnecessary use of NoImplicitPrelude pragma (#7089) (Albert Krewinkel) @@ -353,6 +430,13 @@ + Clean up benchmark code. + Allow specifying patterns using `-p blah'. + * trypandoc: add 2 second timeout. + + * Use `-split-sections` in creating linux release binary. + This reduces executable size significantly (by about 30%). + + * Remove `weigh-pandoc`. It's not really useful any more, now that our + regular benchmarks include data on allocation. ## pandoc 2.11.4 (2021-01-22) |