aboutsummaryrefslogtreecommitdiff
path: root/src/Text/Pandoc/Readers/HTML.hs
AgeCommit message (Collapse)AuthorFilesLines
2010-12-30New HTML reader using tagsoup as a lexer.John MacFarlane1-582/+379
* The new reader is faster and more accurate. * API changes for Text.Pandoc.Readers.HTML: - removed rawHtmlBlock, anyHtmlBlockTag, anyHtmlInlineTag, anyHtmlTag, anyHtmlEndTag, htmlEndTag, extractTagType, htmlBlockElement, htmlComment - added htmlTag, htmlInBalanced, isInlineTag, isBlockTag, isTextTag * tagsoup is a new dependency. * Text.Pandoc.Parsing: Generalized type on readWith. * Benchmark.hs: Added length calculation to force full evaluation. * Updated HTML reader tests. * Updated markdown and textile readers to use the functions from the HTML reader. * Note: The markdown reader now correctly handles some cases it did not before. For example: <hr/> is reproduced without adding a space. <script> a = '<b>'; </script> is parsed correctly.
2010-12-22HTML reader: Simplified parsing of <script> sections.John MacFarlane1-24/+1
I had previously assumed that we needed to ignore </script> occuring in a string literal or javascript comment. It turns out, though, that browsers aren't that smart.
2010-12-22Made --smart work with HTML reader.John MacFarlane1-4/+13
It did not work before, because - and quotes were gobbled up by the str parser.
2010-12-15HTML reader: allow : in tags.John MacFarlane1-2/+6
Resolves Issue #274.
2010-12-10Removed HTML sanitization.John MacFarlane1-90/+5
This is better done on the resulting HTML; use the xss-sanitize library for this. xss-sanitize is based on pandoc's sanitization, but improves it. - Removed stateSanitize from ParserState. - Removed --sanitize-html option.
2010-12-07Make --smart work in HTML reader.John MacFarlane1-2/+3
2010-12-03Basic Textile Readerpaul.rivier1-1/+2
2010-11-11HTML reader: don't parse raw HTML inside <code> tag.John MacFarlane1-2/+2
Previously '<code><a>x</a></code>' would be parsed as Code "<a>x</a>", which is not what you want.
2010-07-14HTML reader: code cleanup + parse <tt> as Code.John MacFarlane1-34/+47
Partially resolves Issue #247.
2010-07-05Moved parsing functions from Text.Pandoc.Shared to new module.John MacFarlane1-1/+2
+ Text.Pandoc.Parsing
2010-03-23Properly escape URIs in all readers.John MacFarlane1-3/+3
2010-03-23Updated copyright notices.John MacFarlane1-2/+2
2010-03-23Fixed treatment of unicode characters in URIs.John MacFarlane1-1/+1
* Added stringToURI to Shared. This is used in the HTML writer for all URIs. It properly URI-encodes high characters (> 127), leaving everything else (including symbols and spaces) the same. * Modified unsanitaryURI to allow UTF8 characters in a URI. (First, we convert the URI to URI-encoded octets, then we pass through parseURIReference.) This resolves gitit Issue #99. Previously '[abc](http://gitit.net/测试)' would not be rendered as a link when --sanitize was selected.
2010-02-12HTML reader: handle spaces before <html>.fiddlosopher1-0/+1
Resolves Issue #216. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1837 788f1e2b-df1e-0410-8736-df70ead52e1b
2010-02-12HTML reader: Be forgiving in parsing a bare list within a list.fiddlosopher1-2/+6
The following is not valid xhtml, but the intent is clear: <ol> <li>one</li> <ol><li>sub</li></ol> <li>two</li> </ol> We'll treat the <ol> as if it's in a <li>. Resolves Issue #215. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1836 788f1e2b-df1e-0410-8736-df70ead52e1b
2010-02-02Made HTML reader much more forgiving.fiddlosopher1-29/+106
+ Incorporated idea (from HXT) that an element can be closed by an open tag for another element. + Javascript is partially parsed to make sure that a <script> section is not closed by a </script> in a comment or string. + More lenient non-quoted attribute values. Now we accept anything but a space character, quote, or <>. This helps in parsing e.g. www.google.com! + Bare & signs are now parsed as a string. This is a common HTML mistake. + Skip a bare < in malformed HTML. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1825 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-12-31Removed redundant imports (found by ghc 6.12).fiddlosopher1-1/+1
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1750 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-12-31Changed Meta author and date types to Inline lists instead of Strings.fiddlosopher1-4/+4
Meta [Inline] [[Inline]] [Inline] rather than Meta [Inline] [String] String. This is a breaking change for libraries that use pandoc and manipulate the metadata. Changed .native files in test suite for new Meta format. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1699 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-12-07Improved syntax for markdown definition lists.fiddlosopher1-2/+2
Definition lists are now more compatible with PHP Markdown Extra. Resolves Issue #24. + You can have multiple definitions for a term (but still not multiple terms). + Multi-block definitions no longer need a column before each block (indeed, this will now cause multiple definitions). + The marker no longer needs to be flush with the left margin, but can be indented at or two spaces. Also, ~ as well as : can be used as the marker (this suggestion due to David Wheeler.) + There can now be a blank line between the term and the definitions. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1656 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-12-05Added "head" to list of HTML block-level tags.fiddlosopher1-1/+1
Resolves Issue #108. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1645 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-11-21Fixed htmlComment parser.fiddlosopher1-1/+1
(Added a needed try.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@1621 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-11-01Properly handle commented-out list items in markdown.fiddlosopher1-0/+1
Example: - a <!-- - b --> - c Resolves Issue #142. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1615 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-10-04Added haddock comments warning that readers assume \n line endings.fiddlosopher1-1/+1
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1608 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-07-21Fixed bug in HTML comment parser.fiddlosopher1-2/+2
Resolves Issue #157. ('try' in the wrong place.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@1605 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-04-29Made htmlComment parser more efficient.fiddlosopher1-1/+3
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1567 788f1e2b-df1e-0410-8736-df70ead52e1b
2009-01-24Moved all haskell source to src subdirectory.fiddlosopher1-0/+675
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1528 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-11-29Moved everything from src into the top-level directory.fiddlosopher1-496/+0
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1104 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-11-03Reverted back to state as of r1062. The template haskell changesfiddlosopher1-0/+496
are more trouble than they're worth. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1064 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-11-03Use template haskell to avoid the need for templates:fiddlosopher1-496/+0
+ Added library Text.Pandoc.Include, with a template haskell function $(includeStrFrom fname) to include a file as a string constant at compile time. + This removes the need for the 'templates' directory or Makefile target. These have been removed. + The base source directory has been changed from src to . + A new 'data' directory has been added, containing the ASCIIMathML.js script, writer headers, and S5 files. + The src/wrappers directory has been moved to 'wrappers'. + The Text.Pandoc.ASCIIMathML library is no longer needed, since Text.Pandoc.Writers.HTML can use includeStrFrom to include the ASCIIMathML.js code directly. It has been removed. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1063 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-09-17Remove just one leading and one trailing newlinefiddlosopher1-3/+11
from contents of <pre>...</pre> in codeBlock parser. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1023 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-09-17Changed parsing of code blocks in HTML reader:fiddlosopher1-7/+8
+ <code> tag is no longer needed. <pre> suffices. + all HTML tags in the code block (e.g. for syntax highlighting) are skipped, because they are not portable to other output formats. git-svn-id: https://pandoc.googlecode.com/svn/trunk@1022 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-09-15Simplified HTML attribute parsing (HTML reader).fiddlosopher1-10/+5
git-svn-id: https://pandoc.googlecode.com/svn/trunk@1016 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-09-14Fixed two bugs in HTML reader:fiddlosopher1-11/+4
+ <code>...</code> not surrounded by <pre> should count as inline HTML, not code block. + parser for minimized attributes should not swallow trailing spaces git-svn-id: https://pandoc.googlecode.com/svn/trunk@1015 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-19Added a necessary "try" in definition of "para"fiddlosopher1-1/+2
(HTML reader). git-svn-id: https://pandoc.googlecode.com/svn/trunk@863 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-18Bug fixes in readers:fiddlosopher1-6/+18
+ LaTeX reader: skip anything after \end{document} + HTML reader: fixed bug skipping material after </html> -- previously, stuff at the end was skipped even if no </html> was present, which meant only part of the file would be parsed and no error issued + HTML reader: added new constant eitherBlockOrInline with elements that may count either as block-level or inline + Modified isInline and isBlock to take this into account + modified rawHtmlBlock to accept any tag (even an inline tag); this is innocuous, because rawHtmlBlock is tried only if a regular inline element can't be parsed. git-svn-id: https://pandoc.googlecode.com/svn/trunk@862 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15Allow htmlComments as rawHtmlInline in HTML reader.fiddlosopher1-2/+3
git-svn-id: https://pandoc.googlecode.com/svn/trunk@844 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-15Major code cleanup in all modules. (Removed unneeded imports,fiddlosopher1-202/+160
reformatted, etc.) More major changes are documented below: + Removed Text.Pandoc.ParserCombinators and moved all its definitions to Text.Pandoc.Shared. + In Text.Pandoc.Shared: - Removed unneeded 'try' in blanklines. - Removed endsWith function and rewrote functions to use isSuffixOf instead. - Added >>~ combinator. - Rewrote stripTrailingNewlines, removeLeadingSpaces. + Moved Text.Pandoc.Entities -> Text.Pandoc.CharacterReferences. - Removed unneeded functions charToEntity, charToNumericalEntity. - Renamed functions using proper terminology (character references, not entities). decodeEntities -> decodeCharacterReferences, characterEntity -> characterReference. - Moved escapeStringToXML to Docbook writer, which is the only thing that uses it. - Removed old entity parser in HTML and Markdown readers; replaced with new charRef parser in Text.Pandoc.Shared. + Fixed accent bug in Text.Pandoc.Readers.LaTeX: \^{} now correctly parses as a '^' character. + Text.Pandoc.ASCIIMathML is no longer an exported module. git-svn-id: https://pandoc.googlecode.com/svn/trunk@835 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-08-08Major change in the way ordered lists are handled:fiddlosopher1-2/+17
+ The changes are documented in README, under Lists. + The OrderedList block element now stores information about list number style, list number delimiter, and starting number. + The readers parse this information, when possible. + The writers use this information to style ordered lists. + Test suites have been changed accordingly. Motivation: It's often useful to start lists with numbers other than 1, and to have control over the style of the list. Added to Text.Pandoc.Shared: + camelCaseToHyphenated + toRomanNumeral + anyOrderedListMarker + orderedListMarker + orderedListMarkers Added to Text.Pandoc.ParserCombinators: + charsInBalanced' + withHorizDisplacement + romanNumeral RST writer: + Force blank line before lists, so that sublists will be handled correctly. LaTeX reader: + Fixed bug in parsing of footnotes containing multiple paragraphs, introduced by use of charsInBalanced. Fix: use charsInBalanced' instead. LaTeX header: + use mathletters option in ucs package, so that basic unicode Greek letters will work properly. git-svn-id: https://pandoc.googlecode.com/svn/trunk@834 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23In HTML reader, filter Nulls in lists of blocks. (These canfiddlosopher1-2/+2
be caused by raw HTML when the parse-raw option isn't selected.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@787 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23Fixed bug in spanStrikeout: case was not exhaustive.fiddlosopher1-1/+1
git-svn-id: https://pandoc.googlecode.com/svn/trunk@786 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23README: Removed the statement that the RST reader doesn't parsefiddlosopher1-0/+1
definition lists. HTML reader: Added failIfStrict to the definitionList parser, so definition lists will be passed through as raw HTML if --strict specified. git-svn-id: https://pandoc.googlecode.com/svn/trunk@783 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-23Added definition list support to HTML reader.fiddlosopher1-13/+32
Added a test for definition lists to the html-reader test suite. git-svn-id: https://pandoc.googlecode.com/svn/trunk@781 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-21Added support for Strikeout, Superscript, and Subscript tofiddlosopher1-1/+26
HTML reader. Thanks to Bradley Sif for the patch for Strikeout (Issue #18). git-svn-id: https://pandoc.googlecode.com/svn/trunk@753 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-12HTML reader: haddock comment fix.fiddlosopher1-1/+1
git-svn-id: https://pandoc.googlecode.com/svn/trunk@690 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-09Moved Text.ParserCombinators.Pandoc ->fiddlosopher1-1/+1
Text.Pandoc.ParserCombinators. This way, all the Pandoc modules are in one place. git-svn-id: https://pandoc.googlecode.com/svn/trunk@663 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-07-07Adjusted copyright notices to 2006-7; usefiddlosopher1-3/+3
real email address instead of lamely attempting to obfuscate. git-svn-id: https://pandoc.googlecode.com/svn/trunk@640 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-04-10Extensive changes stemming from a rethinking of the Pandoc datafiddlosopher1-14/+5
structure. Key and Note blocks have been removed. Link and image URLs are now stored directly in Link and Image inlines, and note blocks are stored in Note inlines. This requires changes in both parsers and writers. Markdown and RST parsers need to extract data from key and note blocks and insert them into the relevant inline elements. Other parsers can be simplified, since there is no longer any need to construct separate key and note blocks. Markdown, RST, and HTML writers need to construct lists of notes; Markdown and RST writers need to construct lists of link references (when the --reference-links option is specified); and the RST writer needs to construct a list of image substitution references. All writers have been rewritten to use the State monad when state is required. This rewrite yields a small speed boost and considerably cleaner code. * Text/Pandoc/Definition.hs: + blocks: removed Key and Note + inlines: removed NoteRef, added Note + modified Target: there is no longer a 'Ref' target; all targets are explicit URL, title pairs * Text/Pandoc/Shared.hs: + Added 'Reference', 'isNoteBlock', 'isKeyBlock', 'isLineClump', used in some of the readers. + Removed 'generateReference', 'keyTable', 'replaceReferenceLinks', 'replaceRefLinksBlockList', along with some auxiliary functions used only by them. These are no longer needed, since reference links are resolved in the Markdown and RST readers. + Moved 'inTags', 'selfClosingTag', 'inTagsSimple', and 'inTagsIndented' to the Docbook writer, since that is now the only module that uses them. + Changed name of 'escapeSGMLString' to 'escapeStringForXML' + Added KeyTable and NoteTable types + Removed fields from ParserState; 'stateKeyBlocks', 'stateKeysUsed', 'stateNoteBlocks', 'stateNoteIdentifiers', 'stateInlineLinks'. Added 'stateKeys' and 'stateNotes'. + Added clause for Note to 'prettyBlock'. + Added 'writerNotes', 'writerReferenceLinks' fields to WriterOptions. * Text/Pandoc/Entities.hs: Renamed 'escapeSGMLChar' and 'escapeSGMLString' to 'escapeCharForXML' and 'escapeStringForXML' * Text/ParserCombinators/Pandoc.hs: Added lineClump parser: parses a raw line block up to and including following blank lines. * Main.hs: Replaced --inline-links with --reference-links. * README: + Documented --reference-links and removed description of --inline-links. + Added note that footnotes may occur anywhere in the document, but must be at the outer level, not embedded in block elements. * man/man1/pandoc.1, man/man1/html2markdown.1: Removed --inline-links option, added --reference-links option * Markdown and RST readers: + Rewrote to fit new Pandoc definition. Since there are no longer Note or Key blocks, all note and key blocks are parsed on a first pass through the document. Once tables of notes and keys have been constructed, the remaining parts of the document are reassembled and parsed. + Refactored link parsers. * LaTeX and HTML readers: Rewrote to fit new Pandoc definition. Since there are no longer Note or Key blocks, notes and references can be parsed in a single pass through the document. * RST, Markdown, and HTML writers: Rewrote using state monad new Pandoc and definition. State is used to hold lists of references footnotes to and be printed at the end of the document. * RTF and LaTeX writers: Rewrote using new Pandoc definition. (Because of the different treatment of footnotes, the "notes" parameter is no longer needed in the block and inline conversion functions.) * Docbook writer: + Moved the functions 'attributeList', 'inTags', 'selfClosingTag', 'inTagsSimple', 'inTagsIndented' from Text/Pandoc/Shared, since they are now used only by the Docbook writer. + Rewrote using new Pandoc definition. (Because of the different treatment of footnotes, the "notes" parameter is no longer needed in the block and inline conversion functions.) * Updated test suite * Throughout: old haskell98 module names replaced by hierarchical module names, e.g. List by Data.List. * debian/control: Include libghc6-xhtml-dev instead of libghc6-html-dev in "Build-Depends." * cabalize: + Remove haskell98 from BASE_DEPENDS (since now the new hierarchical module names are being used throughout) + Added mtl to BASE_DEPENDS (needed for state monad) + Removed html from GHC66_DEPENDS (not needed since xhtml is now used) git-svn-id: https://pandoc.googlecode.com/svn/trunk@580 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-03Made image parsing in HTML reader sensitive to thefiddlosopher1-3/+6
--inline-links option. git-svn-id: https://pandoc.googlecode.com/svn/trunk@556 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-03-03Added --inline-links option to force links in HTML to be parsedfiddlosopher1-3/+6
as inline links, rather than reference links. (Addresses Issue #4.) git-svn-id: https://pandoc.googlecode.com/svn/trunk@554 788f1e2b-df1e-0410-8736-df70ead52e1b
2007-01-28+ Simplified entity handling by removing stringToSGML from Entities.hs.fiddlosopher1-1/+3
It is no longer needed now that all entities are processed in the markdown and HTML readers. All calls to stringToSGML have been replaced by calls to encodeEntities. + Since inTag's attribute handling already encodes entities, calls to encodeEntities are no longer needed for attribute values, so they've been removed. + The HTML and Markdown readers now call decodeEntities on all raw strings (e.g. authors, dates, link titles), to ensure that no unprocessed entities are included in the native representation of the document. (In the HTML reader, most of this work is done by a change in extractAttributeName.) + The result is a small speed improvement (around 5% on my benchmark) and cleaner code. git-svn-id: https://pandoc.googlecode.com/svn/trunk@519 788f1e2b-df1e-0410-8736-df70ead52e1b
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-02 16:06:43 +0000