aboutsummaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Readers/Markdown.hs
AgeCommit message (Collapse)AuthorFilesLines
2017-09-16Markdown reader: added inlines, inlines1.John MacFarlane1-18/+23
Eventually we'll add `processEmphasis` and `processBracketed` to this. This will allow us to conform to CommonMark rules and fix #3903 and #1735.
2017-08-19Markdown reader: use CommonMark rules for list item nesting.John MacFarlane1-68/+61
Closes #3511. Previously pandoc used the four-space rule: continuation paragraphs, sublists, and other block level content had to be indented 4 spaces. Now the indentation required is determined by the first line of the list item: to be included in the list item, blocks must be indented to the level of the first non-space content after the list marker. Exception: if are 5 or more spaces after the list marker, then the content is interpreted as an indented code block, and continuation paragraphs must be indented two spaces beyond the end of the list marker. See the CommonMark spec for more details and examples. Documents that adhere to the four-space rule should, in most cases, be parsed the same way by the new rules. Here are some examples of texts that will be parsed differently: - a - b will be parsed as a list item with a sublist; under the four-space rule, it would be a list with two items. - a code Here we have an indented code block under the list item, even though it is only indented six spaces from the margin, because it is four spaces past the point where a continuation paragraph could begin. With the four-space rule, this would be a regular paragraph rather than a code block. - a code Here the code block will start with two spaces, whereas under the four-space rule, it would start with `code`. With the four-space rule, indented code under a list item always must be indented eight spaces from the margin, while the new rules require only that it be indented four spaces from the beginning of the first non-space text after the list marker (here, `a`). This change was motivated by a slew of bug reports from people who expected lists to work differently (#3125, #2367, #2575, #2210, #1990, #1137, #744, #172, #137, #128) and by the growing prevalance of CommonMark (now used by GitHub, for example). Users who want to use the old rules can select the `four_space_rule` extension. * Added `four_space_rule` extension. * Added `Ext_four_space_rule` to `Extensions`. * `Parsing` now exports `gobbleAtMostSpaces`, and the type of `gobbleSpaces` has been changed so that a `ReaderOptions` parameter is not needed.
2017-08-18Markdown reader: fixed parsing of fenced code after list...John MacFarlane1-1/+4
...when there is no intervening blank line. Closes #3733.
2017-08-18Markdown reader: parse `-@roe` as suppress-author citation.John MacFarlane1-2/+4
Previously only `[-@roe]` (with brackets) was recognized as suppress-author, and `-@roe` was treated the same as `@roe`. Closes jgm/pandoc-citeproc#237.
2017-08-07Markdown reader: fixed spurious parsing as citation as reference def.John MacFarlane1-2/+4
We now disallow reference keys starting with `@` if the `citations` extension is enabled. Closes #3840.
2017-07-24LaTeX reader: Removed 'macro'.John MacFarlane1-10/+2
It is no longer necessary, since the rawLaTeXBlock parser will parse macro definitions. This also avoids the need for a separate latexMacro parser in the Markdown reader.
2017-07-07Rewrote LaTeX reader with proper tokenization.John MacFarlane1-9/+10
This rewrite is primarily motivated by the need to get macros working properly. A side benefit is that the reader is significantly faster (27s -> 19s in one benchmark, and there is a lot of room for further optimization). We now tokenize the input text, then parse the token stream. Macros modify the token stream, so they should now be effective in any context, including math. Thus, we no longer need the clunky macro processing capacities of texmath. A custom state LaTeXState is used instead of ParserState. This, plus the tokenization, will require some rewriting of the exported functions rawLaTeXInline, inlineCommand, rawLaTeXBlock. * Added Text.Pandoc.Readers.LaTeX.Types (new exported module). Exports Macro, Tok, TokType, Line, Column. [API change] * Text.Pandoc.Parsing: adjusted type of `insertIncludedFile` so it can be used with token parser. * Removed old texmath macro stuff from Parsing. Use Macro from Text.Pandoc.Readers.LaTeX.Types instead. * Removed texmath macro material from Markdown reader. * Changed types for Text.Pandoc.Readers.LaTeX's rawLaTeXInline and rawLaTeXBlock. (Both now return a String, and they are polymorphic in state.) * Added orgMacros field to OrgState. [API change] * Removed readerApplyMacros from ReaderOptions. Now we just check the `latex_macros` reader extension. * Allow `\newcommand\foo{blah}` without braces. Fixes #1390. Fixes #2118. Fixes #3236. Fixes #3779. Fixes #934. Fixes #982.
2017-06-30Make `east_asian_line_breaks` affect all readers/writers.John MacFarlane1-2/+1
Closes #3703.
2017-06-27Require nonempty alt text for `implicit_figures`.John MacFarlane1-1/+2
A figure with an empty caption doesn't make sense. Closes #2844.
2017-06-23Added comment in source.John MacFarlane1-0/+3
2017-06-23Markdown reader: interpret YAML metadata as Inlines when possible.John MacFarlane1-12/+13
If the metadata field is all on one line, we try to interpret it as Inlines, and only try parsing as Blocks if that fails. If it extends over one line (including possibly the `|` or `>` character signaling an indented block), then we parse as Blocks. This was motivated by some German users finding that date: '22. Juin 2017' got parsed as an ordered list. Closes #3755.
2017-06-23Text.Pandoc.Extensions: Added `Ext_raw_attribute`.John MacFarlane1-9/+33
Documented in MANUAL.txt. This is enabled by default in pandoc markdown and multimarkdown.
2017-06-20Removed an 'error' bomb.John MacFarlane1-1/+4
2017-06-20Move CR filtering from tabFilter to the readers.John MacFarlane1-1/+1
The readers previously assumed that CRs had been filtered from the input. Now we strip the CRs in the readers themselves, before parsing. (The point of this is just to simplify the parsers.) Shared now exports a new function `crFilter`. [API change] And `tabFilter` no longer filters CRs.
2017-06-19Separated tracing from logging.John MacFarlane1-4/+2
Formerly tracing was just log messages with a DEBUG log level. We now make these things independent. Tracing can be turned on or off in PandocMonad using `setTrace`; it is independent of logging. * Removed `DEBUG` from `Verbosity`. * Removed `ParserTrace` from `LogMessage`. * Added `trace`, `setTrace` to `PandocMonad`.
2017-06-11Don't allow backslash + newline to affect block structure.John MacFarlane1-3/+8
Note that as a result of this change, the following, which formerly produced a header with two lines separated by a line break, will now produce a header followed by a paragraph: # Hi\ there This may affect some existing documents that relied on this undocumented and unintended behavior. This change makes pandoc more consistent with other Markdown implementations, and with itself (since the two-space version of a line break doesn't work inside ATX headers, and neither version works inside Setext headers). Closes #3730.
2017-06-10Changed all readers to take Text instead of String.John MacFarlane1-2/+3
Readers: Renamed StringReader -> TextReader. Updated tests. API change.
2017-05-30Added eastAsianLineBreakFilter to Shared.John MacFarlane1-11/+1
This used to live in the Markdown reader.
2017-05-28Markdown reader: use anyLineNewlineAlexander Krotov1-4/+3
2017-05-27Markdown writer: changes to `--reference-links`.John MacFarlane1-2/+6
With `--reference-location` of `section` or `block`, pandoc will now repeat references that have been used in earlier sections. The Markdown reader has also been modified, so that *exactly* repeated references do not generate a warning, only references with the same label but different targets. The idea is that, with references after every block, one might want to repeat references sometimes. Closes #3701.
2017-05-25Added `spaced_reference_links` extension.John MacFarlane1-3/+5
This is now the default for pandoc's Markdown. It allows whitespace between the two parts of a reference link: e.g. [a] [b] [b]: url This is now forbidden by default. Closes #2602.
2017-05-25Markdown reader: warn for notes defined but not used.John MacFarlane1-6/+14
Closes #1718. Parsing.ParserState: Make stateNotes' a Map, add stateNoteRefs.
2017-05-24Markdown reader: fixed smart quotes after emphasis.John MacFarlane1-5/+6
E.g. in *foo*'s 'foo' Closes #2228.
2017-05-24Parsing: Provide parseFromString'.John MacFarlane1-19/+21
This is a verison of parseFromString specialied to ParserState, which resets stateLastStrPos at the end. This is almost always what we want. This fixes a bug where `_hi_` wasn't treated as emphasis in the following, because pandoc got confused about the position of the last word: - [o] _hi_ Closes #3690.
2017-05-18Markdown: allow attributes in reference links to start on next line.John MacFarlane1-1/+3
This addresses a subsidiary issue in #3674.
2017-05-17Move anyLineNewline to Parsing.hsAlexander Krotov1-1/+1
2017-05-13Update dates in copyright noticesAlbert Krewinkel1-2/+2
This follows the suggestions given by the FSF for GPL licensed software. <https://www.gnu.org/prep/maintain/html_node/Copyright-Notices.html>
2017-05-11Combine grid table parsersAlbert Krewinkel1-83/+1
The grid table parsers for markdown and rst was combined into one single parser, slightly changing parsing behavior of both parsers: - The markdown parser now compactifies block content cell-wise: pure text blocks in cells are now treated as paragraphs only if the cell contains multiple paragraphs, and as plain blocks otherwise. Before, this was true only for single-column tables. - The rst parser now accepts newlines and multiple blocks in header cells. Closes: #3638
2017-05-06Markdown reader: improved parsing of indented raw HTML blocks.John MacFarlane1-1/+7
Previously we inadvertently interpreted indented HTML as code blocks. This was a regression. We now seek to determine the indentation level of the contents of an HTML block, and (optionally) skip that much indentation. As a side effect, indentation may be stripped off of raw HTML blocks, if `markdown_in_html_blocks` is used. This is better than having things interpreted as indented code blocks. Closes #1841.
2017-04-30Provide shared F monad functions for Markdown and Org readersAlbert Krewinkel1-4/+1
The `F` monads used for delayed evaluation of certain values in the Markdown and Org readers are based on a shared data type capturing the common pattern of both `F` types.
2017-04-06Allow raw latex commands starting with `\start` in Markdown.John MacFarlane1-1/+2
Previously these weren't allowed because they were interpreted as starting ConTeXt environments, even without a corresponding \stop... Closes #3558.
2017-03-22Markdown reader: allow latex macro definitions indented 1-3 spaces.John MacFarlane1-1/+8
Previously they only worked if nonindented.
2017-03-20Add `space_in_atx_header` extension.John MacFarlane1-0/+1
This is enabled by default in pandoc and GitHub markdown but not the other flavors. This requirse a space between the opening #'s and the header text in ATX headers (as CommonMark does but many other implementations do not). This is desirable to avoid falsely capturing things ilke #hashtag or #5 Closes #3512.
2017-03-11Markdown reader: optimized nonindentSpaces.John MacFarlane1-6/+3
Makes the benchmark go from 40 to 36 ms.
2017-03-05Markdown reader: fixed internal header links.John MacFarlane1-2/+7
Closes #2397. This patch also adds `shortcut_reference_links` to the list of mmd extensions.
2017-03-05Added readerAbbreviations to ParserState.John MacFarlane1-25/+16
Markdown reader now consults this to determine what is an abbreviation. Eventually it will be possible to specify a custom list (see #256).
2017-03-05Markdown reader: Fixed regression on left-biased union for metadata.John MacFarlane1-6/+9
When multiple YAML metadata blocks are used, and two define the same field, the value defined first takes precedence, according to the manual. This was changed briefly in ba3ee62323644134f2a5dd3277e3349d3681111a. This commit reverts to the original behavior and adds a test case.
2017-03-05Parse YAML metadata in a context that sees footnotes...John MacFarlane1-84/+83
defined in the body of the document. Closes #1279.
2017-03-04Markdown reader: treat span with class `smallcaps` as SmallCaps.John MacFarlane1-13/+18
This allows users to specify small caps in Markdown this way: [my text]{.smallcaps} See #1592.
2017-03-04Stylish-haskell automatic formatting changes.John MacFarlane1-49/+49
2017-03-02Markdown reader: when splitting pipe table cells, skip tex math.John MacFarlane1-1/+1
You might have a `|` character inside math. (Or for that matter something that the parser might mistake for raw HTML.) See #3481.
2017-02-17Markdown reader: Use logMessage instead of report.John MacFarlane1-5/+7
2017-02-11Use new warnings throughout the code base.John MacFarlane1-14/+11
2017-02-10Added Text.Pandoc.Logging (exported module).John MacFarlane1-0/+1
This now contains the Verbosity definition previously in Options, as well as a new LogMessage datatype that will eventually be used instead of raw strings for warnings. This will enable us, among other things, to provide machine-readable warnings if desired. See #3392.
2017-02-06Removed --parse-raw and readerParseRaw.John MacFarlane1-4/+0
These were confusing. Now we rely on the +raw_tex or +raw_html extension with latex or html input. Thus, instead of --parse-raw -f latex we use -f latex+raw_tex and instead of --parse-raw -f html we use -f html+raw_html
2017-01-27Shared: rename compactify', compactify'DL -> compactify, compactifyDL.John MacFarlane1-4/+4
2017-01-25More logging-related changes.John MacFarlane1-7/+3
Class: * Removed getWarnings, withWarningsToStderr * Added report * Added logOutput to PandocMonad * Make logOutput streaming in PandocIO monad * Properly reverse getLog output Readers: * Replaced use of trace with report DEBUG. TWiki Reader: Put everything inside PandocMonad m. API changes.
2017-01-25Changes to verbosity in writer and reader options.John MacFarlane1-1/+1
API changes: Text.Pandoc.Options: * Added Verbosity. * Added writerVerbosity. * Added readerVerbosity. * Removed writerVerbose. * Removed readerTrace. pandoc CLI: The `--trace` option sets verbosity to DEBUG; the `--quiet` option sets it to ERROR, and the `--verbose` option sets it to INFO. The default is WARNING.
2017-01-25Removed readerSmart and the --smart option; added Ext_smart extension.John MacFarlane1-3/+3
Now you will need to do -f markdown+smart instead of -f markdown --smart This change opens the way for writers, in addition to readers, to be sensitive to +smart, but this change hasn't yet been made. API change. Command-line option change. Updated manual.
2017-01-25Make Extensions a custom type instead of a Set Extension.John MacFarlane1-9/+9
The type is implemented in terms of an underlying bitset which should be more efficient. API change: from Text.Pandoc.Extensions export Extensions, emptyExtensions, extensionsFromList, enableExtension, disableExtension, extensionEnabled.
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-06 09:26:18 +0000