aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--changelog.md280
1 files changed, 182 insertions, 98 deletions
diff --git a/changelog.md b/changelog.md
index b13bf1fd4..a48b668b5 100644
--- a/changelog.md
+++ b/changelog.md
@@ -2,6 +2,11 @@
## pandoc 2.12 (UNRELEASED -- PROVISIONAL)
+ * When downloading content from URL arguments, be sensitive to
+ the character encoding (#5600). We can properly handle UTF-8 and latin1
+ (ISO-8859-1); for others we raise an error. Fall back to latin1 if
+ no charset is given in the mime type and UTF-8 decoding fails.
+
* Add new unexported module Text.Pandoc.XML.Light, as well
as Text.Pandoc.XML.Light.Types, Text.Pantoc.XML.Light.Proc,
Text.Pandoc.XML.Light.Output. (Closes #6001, #6565, #7091).
@@ -34,75 +39,6 @@
these have been added. And the docx golden tests have been updated,
because the new parser does not preserve the order of attributes.
- * Text.Pandoc.App
-
- + Add `parseOptionsFromArgs` [API change, new exported function].
-
- * Text.Pandoc.Citeproc.BibTeX
-
- + `Text.Pandoc.Citeproc.writeBibTeXString` now returns
- `Doc Text` instead of `Text` (#7068).
- + Correctly handle `pages` (= `page` in CSL) (#7067).
- + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067).
- + In BibTeX output, protect foreign titles since there's no language
- field (#7067).
- + Clean up BibTeX parsing (#7049). Previously there was a messy code
- path that gave strange results in some cases, not passing through raw
- tex but trying to extract a string content. This was an artefact of
- trying to handle some special bibtex-specific commands in the BibTeX
- reader. Now we just handle these in the LaTeX reader and simplify
- parsing in the BibTeX reader. This does mean that more raw tex will
- be passed through (and currently this is not sensitive to the
- `raw_tex` extension; this should be fixed).
-
- * Text.Pandoc.Citeproc.MetaValue
-
- + Correctly parse "raw" date value in markdown references metadata.
- (See jgm/citeproc#53.)
-
- * Text.Pandoc.Class
-
- + Add `getTimestamp` [API change]. This attempts to read the
- `SOURCE_DATE_EPOCH` environment variable and parse a UTC time
- from it (treating it as a unix date stamp, see
- https://reproducible-builds.org/specs/source-date-epoch/). If the
- variable is not set or can't be parsed as a unix date stamp, then the
- function returns the current date.
-
- * Text.Pandoc.Error
-
- + Remove unused variables (Albert Krewinkel)
- + Export `renderError` [API change].
- + Refactor `handleError` to use `renderError`. This allows us render
- error messages without exiting.
-
- * Text.Pandoc.Extensions
-
- + `Ext_task_lists` is now supported by org (and turned
- on by default) (Albert Krewinkel, #6336).
- + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes
- (#7097). This attribute was listed as allowed, but it didn't actually
- do anything. Use `attributes` for code attributes and more.
-
- * Lua subsystem:
-
- + Always load built-in Lua scripts from default data-dir (Albert
- Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always
- loaded from the system's default data directory. Loading from a
- different directory by overriding the default path, e.g. via
- `--data-dir`, is no longer supported to avoid unexpected behavior
- and to address security concerns.
- + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565).
- The module allows to work with file paths in a convenient and
- platform-independent manner.
-
- * Text.Pandoc.PDF
-
- + Disable `smart` extension when building PDF via LaTeX.
- This is to prevent accidental creation of ligatures like
- `` ?` `` and `` !` `` (especially in languages with quotations like
- German), and similar ligature issues. (See jgm/citeproc#54.)
-
* DocBook reader:
+ Avoid expensive tree normalization step, as it is not necessary
@@ -137,6 +73,11 @@
* LaTeX reader:
+ + Don't export `tokenize`, `untokenize` [API change]. These are internal
+ implementation details, which were only exported for testing.
+ They don't belong in the public API.
+ + Improved efficiency of the parser. With these changes the reader
+ is almost twice as fast as in the last release in our benchmarks.
+ Code cleanup, removing some unnecessary things.
+ Rewrite `withRaw` so it doesn't rely on fragile assumptions
about token positions (which break when macros are expanded)
@@ -156,11 +97,19 @@
it doesn't change anything.
+ Improve `braced'`. Remove the parameter, have it parse the
opening brace, and make it more efficient.
+ + Factor out pieces of the LaTeX reader to make the module smaller.
+ This reduces memory demands when compiling. Created
+ Text.Pandoc.Readers.{LaTeX,Math,Citation,Table,Macro,Accent}.
+ Changed Text.Pandoc.Readers.LaTeX.SIunitx to export a command map
+ instead of individual commands.
+
+ * Make Text.Pandoc.Readers.LaTeX.Types an unexported module [API change].
* Markdown reader:
+ Improved handling of mmd link attributes in references (#7080).
Previously they only worked for links that had titles.
+ + Improved efficiency of the parser (benchmarks show a 15% speedup).
* OPML reader:
@@ -218,7 +167,7 @@
+ Use `getTimestamp` instead of `getCurrentTime` for timestamp.
Setting `SOURCE_DATE_EPOCH` will allow reproducible builds.
- * Text.Pandoc.Writers.EPUB
+ * EPUB writer:
+ Use `getTimestamp` instead of `getCurrentTime` for timestamp.
Setting `SOURCE_DATE_EPOCH` will allow reproducible builds (#7093).
@@ -233,6 +182,10 @@
contains an ampersand or another character with a special meaning
in XML.
+ * Jira writer:
+
+ + Use Span identifiers as anchors (Albert Krewinkel,
+
* LaTeX writer:
+ Adjust hypertargets to beginnings of paragraphs (#7078).
@@ -241,6 +194,8 @@
This makes a particular difference for links to citations using
`--citeproc` and `link-citations: true`.
+ Change BCP47 lang tag from `jp` to `ja` (Mauro Bieg, #7047).
+ + Use function instead of map for accent lookup (should be
+ more efficient).
* Markdown writer:
@@ -279,40 +234,151 @@
`--include-in-header`, `--include-before-body`, `--include-after-body`)
may be used.
- * LaTeX template: Update to iftex package (#7073) (Andrew Dunning)
+ * LaTeX template:
+
+ + Update to iftex package (#7073) (Andrew Dunning)
+ + Wrap url colours in braces (#7121) (Loïc Grobol).
* revealjs template: Add 'center' option for vertical slide centering.
(maurerle, #7104).
* Text.Pandoc.XML: Improve efficiency of `fromEntities`.
- * Test suite: a more robust way of testing the executable.
- Many of our tests require running the pandoc executable. This is
- problematic for a few different reasons. First, cabal-install will
- sometimes run the test suite after building the library but before
- building the executable, which means the executable isn't in place for
- the tests. One can work around that by first building, then building and
- running the tests, but that's fragile. Second, we have to find the
- executable. So far, we've done that using a function `findPandoc` that
- attempts to locate it relative to the test executable (which can be
- located using findExecutablePath). But the logic here is delicate and
- work with every combination of options. To solve both problems, we add
- an `--emulate` option to the `test-pandoc` executable. When `--emulate`
- occurs as the first argument passed to `test-pandoc`, the program simply
- emulates the regular pandoc executable, using the rest of the arguments
- (after `--emulate`). Thus, `test-pandoc --emulate -f markdown -t latex`
- is just like `pandoc -f markdown -t latex`.
- Since all the work is done by library functions, implementing this
- emulation just takes a couple lines of code and should be entirely
- reliable. With this change, we can test the pandoc executable by running
- the test program itself (locatable using `findExecutablePath`) with the
- `--emulate` option. This removes the need for the fragile `findPandoc`
- step, and it means we can run our integration tests even when we're just
- building the library, not the executable. [Note: part of this change
- involved simplifying some complex handling to set environment variables
- for dynamic library paths. I have tested a build with
- `--enable-dynamic-executable`, and it works, but further testing may be
- needed.]
+ * Text.Pandoc.MIME
+
+ + Add exported function `getCharset` [API change].
+
+ * Text.Pandoc.UTF8: change IO functions to return Text, not String
+ [API change]. This affects `readFile`, `getContents`, `writeFileWith`,
+ `writeFile`, `putStrWith`, `putStr`, `putStrLnWith`, `putStrLn`.
+ `hPutStrWith`, `hPutStr`, `hPutStrLnWith`, `hPutStrLn`, `hGetContents`.
+ This avoids the need to uselessly create a linked list of characters
+ when emiting output.
+
+ * Text.Pandoc.App
+
+ + Add `parseOptionsFromArgs` [API change, new exported function].
+
+ * Text.Pandoc.Citeproc.BibTeX
+
+ + `Text.Pandoc.Citeproc.writeBibTeXString` now returns
+ `Doc Text` instead of `Text` (#7068).
+ + Correctly handle `pages` (= `page` in CSL) (#7067).
+ + Correctly handle BibLaTeX `langid` (= `language` in CSL, #7067).
+ + In BibTeX output, protect foreign titles since there's no language
+ field (#7067).
+ + Clean up BibTeX parsing (#7049). Previously there was a messy code
+ path that gave strange results in some cases, not passing through raw
+ tex but trying to extract a string content. This was an artefact of
+ trying to handle some special bibtex-specific commands in the BibTeX
+ reader. Now we just handle these in the LaTeX reader and simplify
+ parsing in the BibTeX reader. This does mean that more raw tex will
+ be passed through (and currently this is not sensitive to the
+ `raw_tex` extension; this should be fixed).
+
+ * Text.Pandoc.Citeproc.MetaValue
+
+ + Correctly parse "raw" date value in markdown references metadata.
+ (See jgm/citeproc#53.)
+
+ * Text.Pandoc.Citeproc
+
+ + Use https URLs for links (Salim B, #7122).
+
+ * Text.Pandoc.Class
+
+ + Add `getTimestamp` [API change]. This attempts to read the
+ `SOURCE_DATE_EPOCH` environment variable and parse a UTC time
+ from it (treating it as a unix date stamp, see
+ https://reproducible-builds.org/specs/source-date-epoch/). If the
+ variable is not set or can't be parsed as a unix date stamp, then the
+ function returns the current date.
+
+ * Text.Pandoc.Error
+
+ + Add `PandocUnsupportedCharsetError` constructor for
+ `PandocError` [API change].
+ + Export `renderError` [API change].
+ + Refactor `handleError` to use `renderError`. This allows us render
+ error messages without exiting.
+ + Remove unused variables (Albert Krewinkel)
+
+ * Text.Pandoc.Extensions
+
+ + `Ext_task_lists` is now supported by org (and turned
+ on by default) (Albert Krewinkel, #6336).
+ + Remove `Ext_fenced_code_attributes` from allowed commonmark attributes
+ (#7097). This attribute was listed as allowed, but it didn't actually
+ do anything. Use `attributes` for code attributes and more.
+
+ * Lua subsystem:
+
+ + Always load built-in Lua scripts from default data-dir (Albert
+ Krewinkel). The Lua modules `pandoc` and `pandoc.List` are now always
+ loaded from the system's default data directory. Loading from a
+ different directory by overriding the default path, e.g. via
+ `--data-dir`, is no longer supported to avoid unexpected behavior
+ and to address security concerns.
+ + Add module "pandoc.path" (Albert Krewinkel, #6001, #6565).
+ The module allows to work with file paths in a convenient and
+ platform-independent manner.
+ + Use strict evaluation when retrieving AST value from the stack
+ (Albert Krewinkel, #6674).
+
+ * Text.Pandoc.PDF
+
+ + Disable `smart` extension when building PDF via LaTeX.
+ This is to prevent accidental creation of ligatures like
+ `` ?` `` and `` !` `` (especially in languages with quotations like
+ German), and similar ligature issues. (See jgm/citeproc#54.)
+
+ * Text.Pandoc.CSV:
+
+ + Fix parsing of unquoted values (#7112). Previously we didn't allow
+ unescaped quotes in unquoted values, but they are allowed
+ in CSV.
+
+ * Test suite:
+
+ + Use a more robust method for testing the executable. Many
+ of our tests require running the pandoc executable. This
+ is problematic for a few different reasons. First,
+ cabal-install will sometimes run the test suite after
+ building the library but before building the executable,
+ which means the executable isn't in place for the tests.
+ One can work around that by first building, then building
+ and running the tests, but that's fragile. Second, we
+ have to find the executable. So far, we've done that using
+ a function `findPandoc` that attempts to locate it
+ relative to the test executable (which can be located
+ using findExecutablePath). But the logic here is delicate
+ and work with every combination of options. To solve both
+ problems, we add an `--emulate` option to the
+ `test-pandoc` executable. When `--emulate` occurs as the
+ first argument passed to `test-pandoc`, the program simply
+ emulates the regular pandoc executable, using the rest of
+ the arguments (after `--emulate`). Thus, `test-pandoc
+ --emulate -f markdown -t latex` is just like `pandoc -f
+ markdown -t latex`. Since all the work is done by library
+ functions, implementing this emulation just takes a couple
+ lines of code and should be entirely reliable. With this
+ change, we can test the pandoc executable by running the
+ test program itself (locatable using `findExecutablePath`)
+ with the `--emulate` option. This removes the need for the
+ fragile `findPandoc` step, and it means we can run our
+ integration tests even when we're just building the
+ library, not the executable. [Note: part of this change
+ involved simplifying some complex handling to set
+ environment variables for dynamic library paths. I have
+ tested a build with `--enable-dynamic-executable`, and it
+ works, but further testing may be needed.]
+ + Print accurate location if a test fails (Albert
+ Krewinkel). Ensures that tasty-hunit reports the location
+ of the failing test instead of the location of the helper
+ `test` function.
+
+ * Documentation: Update URLs and use `https` where possible (#7122,
+ Salim B).
* MANUAL.txt
@@ -328,6 +394,9 @@
* Makefile: in `make bench`, create CSV files for comparison and compare
against previous benchmark run. Add timestamp to CSV filenames.
+ * cabal.project: don't explicitly set -trypandoc.
+ If we do, this can't be overridden on the cabal command line.
+
* doc/lua-filters.md: improve documentation for
`pandoc.mediabag.insert`, `pandoc.mediabag.fetch`,
`directory`, `normalize` (Albert Krewinkel).
@@ -339,6 +408,14 @@
* Require citeproc 0.3.0.7, which correctly titlecases when titles
contain non-ASCII characters.
+ * Use skylighting 0.10.4. This version of skylighting uses xml-conduit
+ rather than hxt. This speeds up parsing of XML syntax definitions
+ fourfold, and removes four packages from pandoc's dependency graph:
+ hxt-charproperties, hxt-unicode, hxt-regex-xmlschema, hxt.
+
+ * Add script `tools/parseTimings.pl` to help pin down which
+ modules take the most time and memory to compile.
+
* Avoid unnecessary use of NoImplicitPrelude pragma (#7089) (Albert
Krewinkel)
@@ -353,6 +430,13 @@
+ Clean up benchmark code.
+ Allow specifying patterns using `-p blah'.
+ * trypandoc: add 2 second timeout.
+
+ * Use `-split-sections` in creating linux release binary.
+ This reduces executable size significantly (by about 30%).
+
+ * Remove `weigh-pandoc`. It's not really useful any more, now that our
+ regular benchmarks include data on allocation.
## pandoc 2.11.4 (2021-01-22)