Fixed bug in Markdown reader's handling of underscores and other

inline formatting markers inside reference labels:  for example,
in '[A_B]: /url/a_b', the material between underscores was being
parsed as emphasized inlines.


git-svn-id: https://pandoc.googlecode.com/svn/trunk@442 788f1e2b-df1e-0410-8736-df70ead52e1b

11 files changed, 44 insertions, 12 deletions

diff --git a/debian/changelog b/debian/changelog
index 506cac161..682b2e54a 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -73,15 +73,20 @@ pandoc (0.3) unstable; urgency=low
     + Revised inline code parsing in Markdown reader to conform to
       markdown standard.  Now any number of `s can begin inline code,
       which will end with the same number of `s.  For example, to
-      have two backticks as code, write ``` `` ```.
+      have two backticks as code, write ``` `` ```.  Modified Markdown
+      writer accordingly.
     + Modified HTML reader to skip a newline following a <br> tag.
       Otherwise the newline will be treated as a space at the beginning
       of the next line.
     + Fixed bug in text-wrapping routine in Markdown and RST writers.
       Now LineBreaks no longer cause wrapping problems.
-    + Fixed bug with inline Code in Markdown writer.  Now it's guaranteed
-      that enough `'s will be used, depending on the content.
-
+    + Supported hexadecimal numerical entity references as well as
+      decimal ones.
+    + Fixed bug in Markdown reader's handling of underscores and other
+      inline formatting markers inside reference labels:  for example,
+      in '[A_B]: /url/a_b', the material between underscores was being
+      parsed as emphasized inlines.
+ 
   * Made handling of code blocks more consistent.  Previously, some
     readers allowed trailing newlines, while others stripped them.
     Now, all readers strip trailing newlines in code blocks. Writers
diff --git a/src/Text/Pandoc/Readers/Markdown.hs b/src/Text/Pandoc/Readers/Markdown.hs
index 35ceb7807..ba4274c1c 100644
--- a/src/Text/Pandoc/Readers/Markdown.hs
+++ b/src/Text/Pandoc/Readers/Markdown.hs
@@ -696,17 +696,23 @@ endline = try (do
 -- links
 --
 
--- a reference label for a link
-reference = do
+rawLabel = try $ do
   char labelStart
-  notFollowedBy (char noteStart)
   -- allow for embedded brackets:
-  label <- manyTill ((do{res <- reference;
-                         return $ [Str "["] ++ res ++ [Str "]"]}) <|>
-                      count 1 inline)
-                    (char labelEnd)
-  return (normalizeSpaces (concat label))
+  raw <- manyTill (do{res <- rawLabel; return ("[" ++ res ++ "]")} <|> 
+                   count 1 anyChar) (char labelEnd)
+  return $ concat raw 
 
+-- a reference label for a link
+reference = try $ do
+  notFollowedBy (try (do{char labelStart; char noteStart}))
+  raw <- rawLabel
+  oldInput <- getInput
+  setInput raw
+  label <- many inline
+  setInput oldInput
+  return (normalizeSpaces label)
+ 
 -- source for a link, with optional title
 source = try (do 
   char srcStart
diff --git a/tests/testsuite.native b/tests/testsuite.native
index cb60c1922..44c133311 100644
--- a/tests/testsuite.native
+++ b/tests/testsuite.native
@@ -270,6 +270,7 @@ Pandoc (Meta [Str "Pandoc",Space,Str "Test",Space,Str "Suite"] ["John MacFarlane
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title preceded by a tab"),Str "."]
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title with &quot;quotes&quot; in it")]
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title with single quotes")]
+, Para [Link [Str "with",Str "_",Str "underscore"] (Src "/url/with_underscore" "")]
 , Para [Link [Str "Email",Space,Str "link"] (Src "mailto:nobody@nowhere.net" "")]
 , Para [Link [Str "Empty"] (Src "" ""),Str "."]
 , Header 2 [Str "Reference"]
diff --git a/tests/testsuite.txt b/tests/testsuite.txt
index f8bf68526..e65e98193 100644
--- a/tests/testsuite.txt
+++ b/tests/testsuite.txt
@@ -502,6 +502,8 @@ Just a [URL](/url/).
 
 [URL and title](/url/ 'title with single quotes')
 
+[with_underscore](/url/with_underscore)
+
 [Email link](mailto:nobody@nowhere.net)
 
 [Empty]().
diff --git a/tests/writer.docbook b/tests/writer.docbook
index c6c99f9bf..616273ea2 100644
--- a/tests/writer.docbook
+++ b/tests/writer.docbook
@@ -816,6 +816,9 @@ Cat    &amp; 1      \\ \hline
         <ulink url="/url/">URL and title</ulink>
       </para>
       <para>
+        <ulink url="/url/with_underscore">with_underscore</ulink>
+      </para>
+      <para>
         <email>nobody@nowhere.net</email>
       </para>
       <para>
diff --git a/tests/writer.html b/tests/writer.html
index a14ef60d7..2428d0a12 100644
--- a/tests/writer.html
+++ b/tests/writer.html
@@ -648,6 +648,9 @@ Cat    &amp; 1      \\ \hline
   <a href="/url/" title="title with single quotes">URL and title</a>
 </p>
 <p>
+  <a href="/url/with_underscore">with_underscore</a>
+</p>
+<p>
   <script type="text/javascript">
 <!--
 h='&#110;&#x6f;&#x77;&#104;&#x65;&#114;&#x65;&#46;&#110;&#x65;&#116;';a='&#64;';n='&#110;&#x6f;&#98;&#x6f;&#100;&#x79;';e=n+a+h;
diff --git a/tests/writer.latex b/tests/writer.latex
index 12d673059..021a0a29c 100644
--- a/tests/writer.latex
+++ b/tests/writer.latex
@@ -492,6 +492,8 @@ Just a \href{/url/}{URL}.
 
 \href{/url/}{URL and title}
 
+\href{/url/with_underscore}{with\_underscore}
+
 \href{mailto:nobody@nowhere.net}{Email link}
 
 \href{}{Empty}.
diff --git a/tests/writer.markdown b/tests/writer.markdown
index 7d4d68f22..42bab4664 100644
--- a/tests/writer.markdown
+++ b/tests/writer.markdown
@@ -514,6 +514,8 @@ Just a [URL](/url/).
 
 [URL and title](/url/ "title with single quotes")
 
+[with\_underscore](/url/with_underscore)
+
 [Email link](mailto:nobody@nowhere.net)
 
 [Empty]().
diff --git a/tests/writer.native b/tests/writer.native
index cb60c1922..44c133311 100644
--- a/tests/writer.native
+++ b/tests/writer.native
@@ -270,6 +270,7 @@ Pandoc (Meta [Str "Pandoc",Space,Str "Test",Space,Str "Suite"] ["John MacFarlane
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title preceded by a tab"),Str "."]
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title with &quot;quotes&quot; in it")]
 , Para [Link [Str "URL",Space,Str "and",Space,Str "title"] (Src "/url/" "title with single quotes")]
+, Para [Link [Str "with",Str "_",Str "underscore"] (Src "/url/with_underscore" "")]
 , Para [Link [Str "Email",Space,Str "link"] (Src "mailto:nobody@nowhere.net" "")]
 , Para [Link [Str "Empty"] (Src "" ""),Str "."]
 , Header 2 [Str "Reference"]
diff --git a/tests/writer.rst b/tests/writer.rst
index e3eb59612..e276ac647 100644
--- a/tests/writer.rst
+++ b/tests/writer.rst
@@ -616,6 +616,8 @@ Just a `URL`_.
 
 `URL and title`_
 
+`with\_underscore`_
+
 `Email link`_
 
 `Empty`_.
@@ -739,6 +741,7 @@ indented.
 .. _quoted link: http://example.com/?foo=1&bar=2
 .. _URL: /url/
 .. _URL and title: /url/
+.. _with\_underscore: /url/with_underscore
 .. _Email link: mailto:nobody@nowhere.net
 .. _Empty: 
 .. _bar: /url/
diff --git a/tests/writer.rtf b/tests/writer.rtf
index 8d24e927c..de2cb723e 100644
--- a/tests/writer.rtf
+++ b/tests/writer.rtf
@@ -271,6 +271,10 @@ URL and title
 URL and title
 }}}
 \par}
+{\pard \f0 \sa180 \li0 \fi0 {\field{\*\fldinst{HYPERLINK "/url/with_underscore"}}{\fldrslt{\ul
+with_underscore
+}}}
+\par}
 {\pard \f0 \sa180 \li0 \fi0 {\field{\*\fldinst{HYPERLINK "mailto:nobody@nowhere.net"}}{\fldrslt{\ul
 Email link
 }}}