aboutsummaryrefslogtreecommitdiff
path: root/test/Tests/Readers
AgeCommit message (Collapse)AuthorFilesLines
2021-05-15HTML writer: parse `<header>` as a DivAlbert Krewinkel1-5/+9
HTML5 `<header>` elements are treated like `<div>` elements.
2021-05-09Change reader types, allowing better tracking of source positions.John MacFarlane1-2/+2
Previously, when multiple file arguments were provided, pandoc simply concatenated them and passed the contents to the readers, which took a Text argument. As a result, the readers had no way of knowing which file was the source of any particular bit of text. This meant that we couldn't report accurate source positions on errors or include accurate source positions as attributes in the AST. More seriously, it meant that we couldn't resolve resource paths relative to the files containing them (see e.g. #5501, #6632, #6384, #3752). Add Text.Pandoc.Sources (exported module), with a `Sources` type and a `ToSources` class. A `Sources` wraps a list of `(SourcePos, Text)` pairs. [API change] A parsec `Stream` instance is provided for `Sources`. The module also exports versions of parsec's `satisfy` and other Char parsers that track source positions accurately from a `Sources` stream (or any instance of the new `UpdateSourcePos` class). Text.Pandoc.Parsing now exports these modified Char parsers instead of the ones parsec provides. Modified parsers to use a `Sources` as stream [API change]. The readers that previously took a `Text` argument have been modified to take any instance of `ToSources`. So, they may still be used with a `Text`, but they can also be used with a `Sources` object. In Text.Pandoc.Error, modified the constructor PandocParsecError to take a `Sources` rather than a `Text` as first argument, so parse error locations can be accurately reported. T.P.Error: showPos, do not print "-" as source name.
2021-04-29Docx reader: add handling of vml image objects (jgm#4735) (#7257)mbrackeantidot1-0/+4
They represent images, the same way as other images in vml format.
2021-04-28Smarter smart quotes.John MacFarlane1-1/+1
Treat a leading " with no closing " as a left curly quote. This supports the practice, in fiction, of continuing paragraphs quoting the same speaker without an end quote. It also helps with quotes that break over lines in line blocks. Closes #7216.
2021-03-31Treat tabs as spaces in ODT Reader. (#7185)niszet1-0/+1
2021-03-13Jira reader: mark divs created from panels with class "panel".Albert Krewinkel1-0/+6
Closes: tarleb/jira-wiki-markup#2
2021-02-28Remove superfluous imports.John MacFarlane1-2/+0
2021-02-28T.P.Readers.LaTeX: Don't export tokenize, untokenize.John MacFarlane1-16/+1
[API change] These were only exported for testing, which seems the wrong thing to do. They don't belong in the public API and are not really usable as they are, without access to the Tok type which is not exported. Removed the tokenize/untokenize roundtrip test. We put a quickcheck property in the comments which may be used when this code is touched (if it is).
2021-02-22Text.Pandoc.UTF8: change IO functions to return Text, not String.John MacFarlane1-1/+1
[API change] This affects `readFile`, `getContents`, `writeFileWith`, `writeFile`, `putStrWith`, `putStr`, `putStrLnWith`, `putStrLn`. `hPutStrWith`, `hPutStr`, `hPutStrLnWith`, `hPutStrLn`, `hGetContents`. This avoids the need to uselessly create a linked list of characters when emiting output.
2021-02-18Org reader: fix bug in org-ref citation parsing.Albert Krewinkel1-0/+40
The org-ref syntax allows to list multiple citations separated by comma. This fixes a bug that accepted commas as part of the citation id, so all citation lists were parsed as one single citation. Fixes: #7101
2021-02-13Org: support task_lists extensionAlbert Krewinkel1-0/+13
The tasks lists extension is now supported by the org reader and writer; the extension is turned on by default. Closes: #6336
2021-02-12Jira: require jira-wiki-markup 1.3.3Albert Krewinkel1-0/+7
* Modified the Doc parser to skip leading blank lines. This fixes parsing of documents which start with multiple blank lines. (#7095) * Prevent URLs within link aliases to be treated as autolinks. (#6944) Fixes: #7095 Fixes: #6944
2021-02-10Add new unexported module T.P.XMLParser.John MacFarlane1-0/+1
This exports functions that uses xml-conduit's parser to produce an xml-light Element or [Content]. This allows existing pandoc code to use a better parser without much modification. The new parser is used in all places where xml-light's parser was previously used. Benchmarks show a significant performance improvement in parsing XML-based formats (especially ODT and FB2). Note that the xml-light types use String, so the conversion from xml-conduit types involves a lot of extra allocation. It would be desirable to avoid that in the future by gradually switching to using xml-conduit directly. This can be done module by module. The new parser also reports errors, which we report when possible. A new constructor PandocXMLError has been added to PandocError in T.P.Error [API change]. Closes #7091, which was the main stimulus. These changes revealed the need for some changes in the tests. The docbook-reader.docbook test lacked definitions for the entities it used; these have been added. And the docx golden tests have been updated, because the new parser does not preserve the order of attributes. Add entity defs to docbook-reader.docbook. Update golden tests for docx.
2021-02-07Avoid unnecessary use of NoImplicitPrelude pragma (#7089)Albert Krewinkel28-54/+0
2021-01-16Revert "Markdown reader: support GitHub wiki's internal links (#2923) (#6458)"John MacFarlane1-30/+0
This reverts commit 6efd3460a776620fdb93812daa4f6831e6c332ce. Since this extension is designed to be used with GitHub markdown (gfm), we need to implement the parser as a commonmark extension (commonmark-extensions), rather than in pandoc's markdown reader. When that is done, we can add it here.
2021-01-16Markdown reader: support GitHub wiki's internal links (#2923) (#6458)Gautier DI FOLCO1-0/+30
Canges overview: * Add a `Ext_markdown_github_wikilink` constructor to `Extension` [API change]. * Add the parser `githubWikiLink` in `Text.Pandoc.Readers.Markdown` * Add tests.
2021-01-09Org reader: allow multiple pipe chars in todo sequencesAlbert Krewinkel1-0/+10
Additional pipe chars, used to separate "action" state from "no further action" states, are ignored. E.g., for the following sequence, both `DONE` and `FINISHED` are states with no further action required. #+TODO: UNFINISHED | DONE | FINISHED Previously, parsing of the todo sequence failed if multiple pipe chars were included. Closes: #7014
2021-01-08Update copyright notices for 2021 (#7012)Albert Krewinkel24-24/+24
2021-01-03Org reader: mark verbatim code with class "verbatim". (#6998)Dimitri Sabadie1-2/+2
* Replace org-mode’s verbatim from code to codeWith. This adds the `"verbatim"` class so that exporters can apply a specific style on it. For instance, it will be possible for HTML to add a CSS rule for code + verbatim class. * Alter test for org-mode’s verbatim change. See previous commit for further detail on the new implementation.
2021-01-01Org reader: restructure output of captioned code blocksAlbert Krewinkel1-3/+3
The Div wrapper of code blocks with captions now has the class "captioned-content". The caption itself is added as a Plain block inside a Div of class "caption". This makes it easier to write filters which match on captioned code blocks. Existing filters will need to be updated. Closes: #6977
2020-12-05Org reader: preserve targets of spurious linksAlbert Krewinkel1-2/+4
Links with (internal) targets that the reader doesn't know about are converted into emphasized text. Information on the link target is now preserved by wrapping the text in a Span of class `spurious-link`, with an attribute `target` set to the link's original target. This allows to recover and fix broken or unknown links with filters. See: #6916
2020-11-24HTML reader tests: disable round-trip testing for tablesAlbert Krewinkel1-11/+3
Information for cell alignment in a column is not preserved during round-trips.
2020-11-22Org reader: parse `#+LANGUAGE` into `lang` metadata fieldAlbert Krewinkel1-0/+4
Fixes: #6845
2020-11-18Replace org #+KEYWORDS with #+keywordsTEC7-92/+92
As of ~2 years ago, lower case keywords became the standard (though they are handled case insensitive, as always): https://code.orgmode.org/bzg/org-mode/commit/13424336a6f30c50952d291e7a82906c1210daf0 Upper case keywords are exclusive to the manual: - https://orgmode.org/list/871s50zn6p.fsf@nicolasgoaziou.fr/ - https://orgmode.org/list/87tuuw3n15.fsf@nicolasgoaziou.fr/
2020-10-14Fix remaining typos in testsAlbert Krewinkel1-1/+1
See: #6738
2020-10-06DOCX reader: Allow empty dates in comments and tracked changes (#6726)Diego Balseiro1-0/+4
For security reasons, some legal firms delete the date from comments and tracked changes. * Make date optional (Maybe) in tracked changes and comments datatypes * Add tests
2020-09-21Markdown reader: Set citationNoteNum accurately in citations.John MacFarlane1-4/+4
This also changes stateLastNoteNumber -> stateNoteNumber.
2020-09-15LaTeX reader: fix improper empty cell filtering (#6689)Christian Despres1-6/+26
2020-09-13Fix hlint suggestions, update hlint.yaml (#6680)Christian Despres6-6/+6
* Fix hlint suggestions, update hlint.yaml Most suggestions were redundant brackets. Some required LambdaCase. The .hlint.yaml file had a small typo, and didn't ignore camelCase suggestions in certain modules.
2020-08-15[Latex Reader] Fixing issues with \multirow and \multicolumn table cells (#6608)Laurent P. René de Cotret1-4/+13
* Added test to replicate (#6596) * Table cell reader not consuming spaces correctly (#6596) * Prevented wrong nesting of \multicolumn and \multirow table cells (#6603) * Parse empty table cells (#6603) * Support full prototype for multirow macro (#6603) Closes #6603
2020-08-07[Latex Reader] Table cell parser not consuming spaces correctly (#6597)Laurent P. René de Cotret1-0/+7
* Added test to replicate (#6596) * Table cell reader not consuming spaces correctly (#6596)
2020-07-23Col-span and row-span in LaTeX reader (#6470)Laurent P. René de Cotret1-3/+55
Add multirow and multicolumn support in LaTex reader. Partially addresses #6311.
2020-07-01Org reader: respect tables-excluding export settingAlbert Krewinkel1-0/+8
Tables can be removed from the final document with the `#+OPTION: |:nil` export setting.
2020-06-30Org reader: respect export setting disabling footnotesAlbert Krewinkel1-0/+16
Footnotes can be removed from the final document with the `#+OPTION: f:nil` export setting.
2020-06-30Org reader: respect export setting which disables entitiesAlbert Krewinkel1-0/+6
MathML-like entities, e.g., `\alpha`, can be disabled with the `#+OPTION: e:nil` export setting.
2020-06-29Org reader: keep unknown keyword lines as raw orgAlbert Krewinkel1-2/+5
The lines of unknown keywords, like `#+SOMEWORD: value` are no longer read as metadata, but kept as raw `org` blocks. This ensures that more information is retained when round-tripping org-mode files; additionally, this change makes it possible to support non-standard org extensions via filters.
2020-06-29Org reader: unify keyword handlingAlbert Krewinkel1-48/+56
Handling of export settings and other keywords (like `#+LINK`) has been combined and unified.
2020-06-29Org reader: support LATEX_HEADER_EXTRA and HTML_HEAD_EXTRA settingsAlbert Krewinkel1-29/+49
These export settings are treated like their non-extra counterparts, i.e., the values are added to the `header-includes` metadata list.
2020-06-29Org reader: allow multiple #+SUBTITLE export settingsAlbert Krewinkel1-0/+7
The values of all lines are read as inlines and collected in the `subtitle` metadata field.
2020-06-28JATS reader: parse abstract element into metadata field of same name (#6482)Albert Krewinkel1-0/+17
Closes: #6480
2020-06-28Org reader: read `#+INSTITUTE` values as text with markupAlbert Krewinkel1-0/+4
The value is stored in the `institute` metadata field and used in the default beamer presentation template.
2020-06-28Org tests: group export settings test for Org readerAlbert Krewinkel1-74/+79
2020-06-28Org reader: update behavior of author, keywords export settingsAlbert Krewinkel1-20/+35
The behavior of the `#+AUTHOR` and `#+KEYWORD` export settings has changed: Org now allows multiple such lines and adds a space between the contents of each line. Pandoc now always parses these settings as meta inlines; setting values are no longer treated as comma-separated lists. Note that a Lua filter can be used to restore the previous behavior.
2020-06-27Org reader: read description lines as inlinesAlbert Krewinkel1-5/+17
`#+DESCRIPTION` lines are treated as text with markup. If multiple such lines are given, then all lines are read and separated by soft linebreaks. Closes: #6485
2020-06-25Org reader: honor tex export optionAlbert Krewinkel1-0/+73
The `tex` export option can be set with `#+OPTION: tex:nil` and allows three settings: - `t` causes LaTeX fragments to be parsed as TeX or added as raw TeX, - `nil` removes all LaTeX fragments from the document, and - `verbatim` treats LaTeX as text. The default is `t`. Closes: #4070
2020-06-22Use native Underline instead of Span in JiraJohn MacFarlane1-1/+1
2020-06-20Recognize images with uppercase extensionsAlbert Krewinkel1-0/+4
Fixes: #6472
2020-04-28Support new Underline element in readers and writers (#6277)Vaibhav Sagar4-10/+6
Deprecate `underlineSpan` in Shared in favor of `Text.Pandoc.Builder.underline`.
2020-04-19More fixes for round-trip tests of HTML reader.John MacFarlane1-6/+10
We exclude tables that have default widths but non-simple content, as these can't really round-trip.
2020-04-18Fixed round-trip HTML tests.John MacFarlane1-0/+5
Exclude tables with cells with line breaks because they don't currently round-trip. (Table goes from being simple to having explicit widths.)