Age | Commit message (Collapse) | Author | Files | Lines |
|
We now use source positions from the token stream to tell us
how much of the text stream to consume. Getting this to
work required a few other changes to make token source positions
accurate.
Closes #7434.
|
|
Generally we allow optional starred variants of LaTeX commands
(since many allow them, and if we don't accept these explicitly,
ignoring the star usually gives acceptable results). But we
don't want to do this for `\(*\)` and similar cases.
Closes #7340.
|
|
Previously we only supported it in inline contexts; now
we support it in all contexts, including math.
Partially addresses #7299.
|
|
Previously, when multiple file arguments were provided, pandoc
simply concatenated them and passed the contents to the readers,
which took a Text argument.
As a result, the readers had no way of knowing which file
was the source of any particular bit of text. This meant that
we couldn't report accurate source positions on errors or
include accurate source positions as attributes in the AST.
More seriously, it meant that we couldn't resolve resource
paths relative to the files containing them
(see e.g. #5501, #6632, #6384, #3752).
Add Text.Pandoc.Sources (exported module), with a `Sources` type
and a `ToSources` class. A `Sources` wraps a list of `(SourcePos,
Text)` pairs. [API change] A parsec `Stream` instance is provided for
`Sources`. The module also exports versions of parsec's `satisfy` and
other Char parsers that track source positions accurately from a
`Sources` stream (or any instance of the new `UpdateSourcePos` class).
Text.Pandoc.Parsing now exports these modified Char parsers instead of
the ones parsec provides. Modified parsers to use a `Sources` as stream
[API change].
The readers that previously took a `Text` argument have been
modified to take any instance of `ToSources`. So, they may still
be used with a `Text`, but they can also be used with a `Sources`
object.
In Text.Pandoc.Error, modified the constructor PandocParsecError
to take a `Sources` rather than a `Text` as first argument,
so parse error locations can be accurately reported.
T.P.Error: showPos, do not print "-" as source name.
|
|
Also taking this opportunity to note, for the record, that
the commit for #7241 should be marked [API change].
It changes the type of `languagesByExtension` in Highlighting,
adding a parameter for a `SyntaxMap`.
|
|
Languages defined using `--syntax-definition` were not recognized by `languagesByExtension`.
This patch corrects that, allowing the writers to see all custom definitions.
The LaTeX still uses the default syntax map, but that's okay in that context, since
`--syntax-definition` won't create new listings styles.
|
|
Update citeproc test.
|
|
[API change]
Use Lang from UnicodeCollation.Lang instead.
This is a richer implementation of BCP 47.
|
|
|
|
|
|
|
|
Closes #7127.
|
|
This reverts commit b569b0226d4bd5e0699077089d54fb03d4394b7d.
Memory usage improvement in compilation wasn't very significant.
|
|
|
|
Incorporate accentCommands into T.P.Readers.LaTeX.Inline.
|
|
|
|
|
|
|
|
[API change]
This is really an implementation detail that shouldn't be
exposed in the public API.
|
|
|
|
|
|
instead of individual commands.
|
|
[API change]
These were only exported for testing, which seems the
wrong thing to do. They don't belong in the public
API and are not really usable as they are, without access
to the Tok type which is not exported.
Removed the tokenize/untokenize roundtrip test.
We put a quickcheck property in the comments which
may be used when this code is touched (if it is).
|
|
|
|
|
|
In conjunction with other changes this makes the reader
almost twice as fast on our benchmark as it was on Feb. 10.
|
|
|
|
These are handled anyway by regularSymbol.
|
|
|
|
|
|
To help reduce memory demands compiling the main LaTeX reader.
|
|
|
|
|
|
* Rewrote `withRaw` so it doesn't rely on fragile assumptions
about token positions (which break when macros are expanded).
This requires the addition of `sEnableWithRaw` and `sRawTokens`
in `LaTeXState`, and a new combinator `disablingWithRaw` to
disable collecting of raw tokens in certain contexts.
* Add `parseFromToks` to T.P.Readers.LaTeX.Parsing.
* Fix parsing of single character tokens so it doesn't mess
up the new raw token collecting.
* These changes slightly increase allocations and have a small
performance impact, but it's minor.
Closes #7092.
|
|
Previously there was a messy code path that gave strange
results in some cases, not passing through raw tex but
trying to extract a string content. This was an artefact
of trying to handle some special bibtex-specific commands
in the BibTeX reader. Now we just handle these in the
LaTeX reader and simplify parsing in the BibTeX reader.
This does mean that more raw tex will be passed through
(and currently this is not sensitive to the `raw_tex`
extension; this should be fixed).
Closes #7049.
|
|
|
|
Closes #7003.
|
|
when `raw_tex` is not enabled. (When `raw_tex` is enabled,
the whole environment is parsed as a raw block.)
The class name is the name of the environment.
Previously, we just included the contents without the
surrounding Div, but having a record of the environment's
boundaries and name can be useful.
Closes #6997.
|
|
If we put an image in italics, then when rendering to Markdown
we no longer get an implicit figure.
Closes #6925.
|
|
|
|
The contents of the `center` environment are put in a `Div`
with class `center`.
|
|
Improves on 9a40976. Closes #6873.
|
|
in cases where we run into trouble parsing inlines til the
closing `]`, e.g. quotes, we return a plain string with the
option contents. Previously we mistakenly included the brackets
in this string.
Closes #6869.
|
|
|
|
Previously this confused the table parser. Closes #6811.
|
|
(LaTeX reader)
Closes #6802.
|
|
`\acl`, `\aclp`, and capitalized versions of already
supported commands.
Closes #6746.
|
|
|
|
Previously we were just treating it as a string and
ignoring accents and formatting. See #6734.
|
|
|