aboutsummaryrefslogtreecommitdiff
path: root/src/Text
AgeCommit message (Collapse)AuthorFilesLines
2007-01-28+ Simplified entity handling by removing stringToSGML from Entities.hs.fiddlosopher6-43/+25
It is no longer needed now that all entities are processed in the markdown and HTML readers. All calls to stringToSGML have been replaced by calls to encodeEntities. + Since inTag's attribute handling already encodes entities, calls to encodeEntities are no longer needed for attribute values, so they've been removed. + The HTML and Markdown readers now call decodeEntities on all raw strings (e.g. authors, dates, link titles), to ensure that no unprocessed entities are included in the native representation of the document. (In the HTML reader, most of this work is done by a change in extractAttributeName.) + The result is a small speed improvement (around 5% on my benchmark) and cleaner code. git-svn-id: https://pandoc.googlecode.com/svn/trunk@519 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-27Use encodeEntities rather than stringToSGML for contents offiddlosopher2-2/+2
Str inline in Docbook and HTML writers, since now these strings should not contain literal entity references. git-svn-id: https://pandoc.googlecode.com/svn/trunk@518 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-27Cleaned up handling of embedded quotes in link titles.fiddlosopher2-9/+4
Now these are stored as a '"' character, not as '"'. The function escapeLinkTitle in the Markdown writer is unnecessary and was removed. Tests modified accordingly. git-svn-id: https://pandoc.googlecode.com/svn/trunk@517 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-27More changes in entity handling: Instead of using entities for charactersfiddlosopher4-52/+51
above 128 in HTML and Docbook output, we now just use unicode. After all, we're declaring UTF-8 content in the header. This makes the HTML and docbook files produced by pandoc much more readable and editable. Changes to Entities.hs: + Removed specialCharToEntity + Added escapeSGMLChar (which just escapes the basic four, <>&") + Modified encodeEntities and stringToSGML to use escapeSGMLChar + Removed encodeEntitiesNumerical + Rewrote encodeEntities for better performance + Rewrote stringToSGML for better performance git-svn-id: https://pandoc.googlecode.com/svn/trunk@516 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-27Changes in entity handling:fiddlosopher7-141/+133
+ Entities are parsed (and unicode characters returned) in both Markdown and HTML readers. + Parsers characterEntity, namedEntity, decimalEntity, hexEntity added to Entities.hs; these parse a string and return a unicode character. + Changed 'entity' parser in HTML reader to use the 'characterEntity' parser from Entities.hs. + Added new 'entity' parser to Markdown reader, and added '&' as a special character. Adjusted test suite accordingly since now we get 'Str "AT",Str "&",Str "T"' instead of 'Str "AT&T".. + stringToSGML moved to Entities.hs. escapeSGML removed as redundant, given encodeEntities. + stringToSGML, encodeEntities, and specialCharToEntity are given a boolean parameter that causes only numerical entities to be used. This is used in the docbook writer. The HTML writer uses named entities where possible, but not all docbook-consumers know about the named entities without special instructions, so it seems safer to use numerical entities there. + decodeEntities is rewritten in a way that avoids Text.Regex, using the new parsers. + charToEntity and charToNumericalEntity added to Entities.hs. + Moved specialCharToEntity from Shared.hs to Entities.hs. + Removed unneeded 'decodeEntities' from 'str' parser in HTML and Markdown readers. + Removed sgmlHexEntity, sgmlDecimalEntity, sgmlNamedEntity, and sgmlCharacterEntity from Shared.hs. + Modified Docbook writer so that it doesn't rely on Text.Regex for detecting "mailto" links. git-svn-id: https://pandoc.googlecode.com/svn/trunk@515 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Rewrote functions in Text/Pandoc/Shared so as not to use Text.Regex,fiddlosopher1-28/+51
which does not support unicode: - escapePreservingRegex removed - stringToSGML rewritten using Parsec parser - new parsers for SGML character entities - escapeSGML rewritten using specialCharToEntity - new function specialCharToEntity git-svn-id: https://pandoc.googlecode.com/svn/trunk@514 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Changed Markdown autoLink parsing to conform better tofiddlosopher1-6/+6
Markdown.pl's behavior. <google.com> is not treated as a link, but <http://google.com>, <ftp://google.com>, and <mailto:google@google.com> are. git-svn-id: https://pandoc.googlecode.com/svn/trunk@513 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Fixed bug in 'extractTagType' in HTML reader: previousfiddlosopher1-1/+4
version was not skipping / in close tags. git-svn-id: https://pandoc.googlecode.com/svn/trunk@512 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Refactored markdown reader so that Text.Regex is not used.fiddlosopher1-14/+19
Replaced email regex test with a custom email autolink parser (autoLinkEmail). Also replaced 'selfClosingTag' with a custom function 'isSelfClosingTag'. git-svn-id: https://pandoc.googlecode.com/svn/trunk@511 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Fixed a bug in extractTagType in HTML Reader: the previousfiddlosopher1-6/+2
version extracted the attributes, too, which is not wanted. git-svn-id: https://pandoc.googlecode.com/svn/trunk@510 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Fixed bug in HTML attribute parser: now a space isfiddlosopher1-2/+2
required before an attribute. Previously, <a.b> would be parsed as an HTML tag with an attribute! git-svn-id: https://pandoc.googlecode.com/svn/trunk@509 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Modified Markdown writer to use autolinks when possible.fiddlosopher1-6/+10
So, instead of [site.com](site.com) we get <site.com>. Changed test suite accordingly. git-svn-id: https://pandoc.googlecode.com/svn/trunk@508 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Rewrote 'extractTagType' in HTML reader so that it doesn't usefiddlosopher1-5/+7
regexs. git-svn-id: https://pandoc.googlecode.com/svn/trunk@507 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24More smart quote bug fixes:fiddlosopher3-4/+20
+ LaTeX writer now handles consecutive quotes properly: for example, ``\,`hello'\,'' + LaTeX reader now parses '\,' as empty Str + normalizeSpaces function in Shared now removes empty Str elements + Modified tests accordingly git-svn-id: https://pandoc.googlecode.com/svn/trunk@506 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-24Fixed bug in smart quoting: recognize ' in contractions likefiddlosopher1-3/+7
"don't" as not beginning single quoted contexts. git-svn-id: https://pandoc.googlecode.com/svn/trunk@505 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-22Removed 'gsub' entirely and replaced its uses with 'substitute'.fiddlosopher6-13/+5
git-svn-id: https://pandoc.googlecode.com/svn/trunk@501 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-22+ Added a 'substitute' function to Shared.hs. This is a genericfiddlosopher2-8/+18
list function that can be used to substitute one substring for another in a string, like 'gsub' except without regular expressions. + Use 'substitute' instead of 'gsub' in the LaTeX writer. This avoids what appears to be a bug in Text.Regex, whereby "\\^" matches "\350". There seems to be a slight speed improvement as well. (Note: If this works, it would be good to replace other uses of gsub that don't employ regexs with 'substitute'.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@500 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-18Small bug fix to last change, and count "'S" as well as "'s" asfiddlosopher1-1/+1
possessive when followed by non-alphanumeric. git-svn-id: https://pandoc.googlecode.com/svn/trunk@499 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-18More tweaks to smart quote parsing: a ' is not a single quotefiddlosopher1-0/+1
start if followed by 's' and then a non-alphanumeric. (Yes, this is English-centric, I'm afraid. But it does help, and I can't think of a language in which 's' by itself is a word.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@498 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-16Minor tweaks to smart quoting code.fiddlosopher1-4/+3
git-svn-id: https://pandoc.googlecode.com/svn/trunk@497 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-16Fixed bug in smart quote recognition: ' before ) or certainfiddlosopher1-3/+4
other punctuation must not be an open quote. git-svn-id: https://pandoc.googlecode.com/svn/trunk@496 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-16Fixed haddock documentation errors.fiddlosopher2-30/+30
git-svn-id: https://pandoc.googlecode.com/svn/trunk@495 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-15Added support for tables in markdown reader and in LaTeX,fiddlosopher10-10/+295
DocBook, and HTML writers. The syntax is documented in README. Tests have been added to the test suite. git-svn-id: https://pandoc.googlecode.com/svn/trunk@493 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-09Don't use named entities in docbook writer. Instead, usefiddlosopher1-4/+4
numerical entities, for portability across stylesheets. git-svn-id: https://pandoc.googlecode.com/svn/trunk@473 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-08+ Changed 'escapedChar' in Markdown reader so that only thefiddlosopher1-1/+8
characters Markdown escapes are escaped in strict mode. When not in strict mode, Pandoc allows all non-alphanumeric characters to be escaped. + Added documentation of backslash escapes to README. git-svn-id: https://pandoc.googlecode.com/svn/trunk@461 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-07Small improvements to indentSpaces. (Allow combinationsfiddlosopher1-1/+2
of spaces and tabs.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@446 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-07Modified HTML output for Image elements, to conform tofiddlosopher1-3/+3
Markdown.pl: + title attribute comes after alt attribute + title is included even if null git-svn-id: https://pandoc.googlecode.com/svn/trunk@445 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-07Fixed performance problems with '--strict' option:fiddlosopher2-17/+18
+ Replaced skipEndline with "option ' ' newline" where possible. + Replaced "notFollowedBy' header" in definition of endline with a faster but equally accurate test for a folliwng header. + Removed check at the beginning of 'reference' for a noteStart: This is not needed, because note comes before referenceKey in the definition of block. + Replaced check for a following anyHtmlBlockTag in autoLink with a check for anyHtmlTag or anyHtmlEndTag. + Other small code cleanups. git-svn-id: https://pandoc.googlecode.com/svn/trunk@444 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-06Fixed bug in Markdown reader's handling of underscores and otherfiddlosopher1-8/+14
inline formatting markers inside reference labels: for example, in '[A_B]: /url/a_b', the material between underscores was being parsed as emphasized inlines. git-svn-id: https://pandoc.googlecode.com/svn/trunk@442 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-06Added support for hexadecimal entities: e.g. &#xA0AB;fiddlosopher1-6/+6
git-svn-id: https://pandoc.googlecode.com/svn/trunk@441 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-06Fixed serious performance problems with new Markdown reader:fiddlosopher2-13/+44
Instead of using lookahead to determine whether a single quote is an apostrophe, we now use state. Inside single quotes, a ' character won't be recognized as the beginning of a single quote. 'stateQuoteContext' has been added to keep track of this. git-svn-id: https://pandoc.googlecode.com/svn/trunk@437 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-06Merged changes from 'quotes' branch since r431. Smart typographyfiddlosopher11-305/+380
is now handled in the Markdown and LaTeX readers, rather than in the writers. The HTML writer has been rewritten to use the prettyprinting library. git-svn-id: https://pandoc.googlecode.com/svn/trunk@436 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-05Changed Markdown writer so that it does not use the single-bracketfiddlosopher1-1/+1
style of implicit reference link. It now uses [this style][], not [this style]. Reason: only newer, beta versions of Markdown allow the single-bracket style. git-svn-id: https://pandoc.googlecode.com/svn/trunk@419 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-04Fixed small bug in consolidateList: added casefiddlosopher1-0/+1
for (Str a):Space:Space:rest. git-svn-id: https://pandoc.googlecode.com/svn/trunk@418 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-04Refactored SGML escaping functions and "in tag" functions tofiddlosopher3-186/+186
Text/Shared/Pandoc. (escapeSGML, stringToSGML, inTag, inTagSimple, inTagIndented, selfClosingTag) These can be used by both the HTML and Docbook writers. git-svn-id: https://pandoc.googlecode.com/svn/trunk@417 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-04Fixed bug in encodeEntities (characters less than 128, not 127,fiddlosopher1-1/+1
should be encoded). git-svn-id: https://pandoc.googlecode.com/svn/trunk@416 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-04Removed unneeded 'options' parameter from 'indentedInTags' functionfiddlosopher1-17/+17
in Docbook writer. git-svn-id: https://pandoc.googlecode.com/svn/trunk@413 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-04+ Improved text wrapping algorithm in markdown, docbook, and RST writers.fiddlosopher4-11/+20
LineBreaks no longer cause ugly wrapping in Markdown output. + Replaced splitBySpace with the more general, polymorphic function splitBy (in Text/Pandoc/Shared). git-svn-id: https://pandoc.googlecode.com/svn/trunk@411 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-03Modified HTML reader to skip a newline following a <br> tag.fiddlosopher1-0/+1
Otherwise the newline will be treated as a space at the beginning of the next line. git-svn-id: https://pandoc.googlecode.com/svn/trunk@410 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-02Change 'HtmlEntities' module to 'Entities'. Adjusted callingfiddlosopher5-20/+20
code accordingly. git-svn-id: https://pandoc.googlecode.com/svn/trunk@395 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-02Use entities for all characters above 127 in docbook output.fiddlosopher2-12/+17
Though XML tools should support unicode, some people will be using SGML tools, and these do not. Using entities makes the docbook files more portable. Also refactored encodeEntities and charToHtmlEntity in HtmlEntities.hs git-svn-id: https://pandoc.googlecode.com/svn/trunk@394 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-01Changed representation of code blocks to use <screen> andfiddlosopher1-1/+1
escaped characters rather than <programlisting> and CDATA. Reason: XML source more easily editable and readable. git-svn-id: https://pandoc.googlecode.com/svn/trunk@393 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-01Removed a line that was causing a compiler warning in docbookfiddlosopher1-1/+0
writer. The line isn't necessary, since we have a case for every kind of block element. git-svn-id: https://pandoc.googlecode.com/svn/trunk@388 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-01Merged changes from docbook branch since r363.fiddlosopher3-2/+241
git-svn-id: https://pandoc.googlecode.com/svn/trunk@386 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-31Revised inline code parsing in Markdown reader to conform tofiddlosopher1-14/+6
Markdown.pl. Now any number of `'s can begin inline code, which will end with the same number of `'s. For example, to have two backticks as code, write ``` `` ``` git-svn-id: https://pandoc.googlecode.com/svn/trunk@360 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-31Simplified list parsing code in RST reader.fiddlosopher1-7/+8
git-svn-id: https://pandoc.googlecode.com/svn/trunk@356 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-31Cleaned up some code in RST reader.fiddlosopher1-20/+15
git-svn-id: https://pandoc.googlecode.com/svn/trunk@354 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-31Changed Markdown reader so that the first pass, in which a listfiddlosopher2-2/+8
of reference keys is made, is much faster. This gets us a big performance boost. git-svn-id: https://pandoc.googlecode.com/svn/trunk@353 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-31Removed unneeded 'do' block from 'parseBlocks' definitionfiddlosopher1-3/+1
in Markdown reader. git-svn-id: https://pandoc.googlecode.com/svn/trunk@352 788f1e2b-df1e-0410-8736-df70ead52e1b
2006-12-30Merged 'strict' branch from r324. This adds a '--strict'fiddlosopher8-74/+162
option to pandoc, which forces it to stay as close as possible to official Markdown syntax. git-svn-id: https://pandoc.googlecode.com/svn/trunk@347 788f1e2b-df1e-0410-8736-df70ead52e1b
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-16 23:15:43 +0000