diff options
| author | Igor Pashev <pashev.igor@gmail.com> | 2021-07-17 20:19:28 +0200 |
|---|---|---|
| committer | Igor Pashev <pashev.igor@gmail.com> | 2021-07-17 20:19:28 +0200 |
| commit | 8ce817dd4453f35ce92afa531c540554429c7299 (patch) | |
| tree | 90236cdc7e59bdf99b32467b89adcb8c5a0b8e22 | |
| parent | b861c20ff2d7460061e73492e3a945e48ef40bac (diff) | |
| parent | d739fd1eea40de9ded3b4f682c849d3c31eba92c (diff) | |
| download | hakyll-8ce817dd4453f35ce92afa531c540554429c7299.tar.gz | |
Merge branch 'master' of https://github.com/jaspervdj/hakyll
29 files changed, 1584 insertions, 677 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md index a11bab2..ac0bb3a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,27 @@ title: Releases # Releases +## Hakyll 4.14.0.0 (2021-03-14) + +- Add `renderPandocWithTransform` and `renderPandocWithTransformM` (by Norman + Liu) +- Make sure the initial project is writable (by Tobias Bora) +- Bump `pandoc` to 2.11.* +- Bump `file-embed` upper bound to 0.0.14 +- Bump `random` upper bound to 1.2 + +## Hakyll 4.13.4.1 (2020-09-30) + +- Bump `pandoc` to 2.10.* +- Bump upper bound for `template-haskell` to 2.17 +- Bump `QuickCheck` upper bound to 2.15 + +## Hakyll 4.13.4.0 (2020-06-20) + +- Miscellaneous Windows-specific fixes and CI (by Laurent P. René de Cotret) +- Bump upper bound for `cryptonite` to 0.28 +- Bump upper bound for `tasty` to 1.4 + ## Hakyll 4.13.3.0 (2020-04-12) - Fix compilation issue related to `MonadFail` on Windows (by Martín Emanuel) diff --git a/hakyll.cabal b/hakyll.cabal index 74a6c2f..4699235 100644 --- a/hakyll.cabal +++ b/hakyll.cabal @@ -1,299 +1,338 @@ -cabal-version: >=1.10 -name: hakyll -version: 4.13.3.0 -license: BSD3 -license-file: LICENSE -maintainer: Jasper Van der Jeugt <m@jaspervdj.be> -author: Jasper Van der Jeugt <m@jaspervdj.be> -homepage: http://jaspervdj.be/hakyll -bug-reports: http://github.com/jaspervdj/Hakyll/issues -synopsis: A static website compiler library -description: - Hakyll is a static website compiler library. It provides you with the tools to - create a simple or advanced static website using a Haskell DSL and formats - such as markdown or RST. You can find more information, including a tutorial, - on the website: - . - * <http://jaspervdj.be/hakyll> - . - If you seek assistance, there's: - . - * A google group: <http://groups.google.com/group/hakyll> - . - * An IRC channel, @#hakyll@ on freenode - . - Additionally, there's the Haddock documentation in the different modules, - meant as a reference. -category: Web -build-type: Simple -data-files: - example/posts/2015-11-28-carpe-diem.markdown - example/posts/2015-10-07-rosa-rosa-rosam.markdown - example/posts/2015-12-07-tu-quoque.markdown - example/posts/2015-08-12-spqr.markdown - example/site.hs - example/images/haskell-logo.png - example/templates/post-list.html - example/templates/default.html - example/templates/archive.html - example/templates/post.html - example/css/default.css - example/index.html - example/about.rst - example/contact.markdown -data-dir: data -extra-source-files: - CHANGELOG.md - tests/data/embed.html - tests/data/example.md - tests/data/example.md.metadata - tests/data/images/favicon.ico - tests/data/just-meta.html - tests/data/just-meta.html.out - tests/data/partial-helper.html - tests/data/partial.html - tests/data/partial.html.out - tests/data/posts/2010-08-26-birthday.md - tests/data/posts/2018-09-26.md - tests/data/posts/2019/05/10/tomorrow.md - tests/data/russian.md - tests/data/strip.html - tests/data/strip.html.out - tests/data/template.html - tests/data/template.html.out - data/templates/atom-item.xml - data/templates/atom.xml - data/templates/rss-item.xml - data/templates/rss.xml - -source-repository head - type: git - location: git://github.com/jaspervdj/hakyll.git - -flag previewserver - description: - Include the preview server - -flag watchserver - description: - Include the watch server - -flag checkexternal - description: - Include external link checking - -flag buildwebsite - description: - Build the hakyll website - default: False - manual: True - -flag usepandoc - description: - Include Pandoc support - manual: True - -library - exposed-modules: - Hakyll - Hakyll.Commands - Hakyll.Core.Compiler - Hakyll.Core.Compiler.Internal - Hakyll.Core.Configuration - Hakyll.Core.Dependencies - Hakyll.Core.File - Hakyll.Core.Identifier - Hakyll.Core.Identifier.Pattern - Hakyll.Core.Item - Hakyll.Core.Logger - Hakyll.Core.Metadata - Hakyll.Core.Provider - Hakyll.Core.Provider.Metadata - Hakyll.Core.Routes - Hakyll.Core.Rules - Hakyll.Core.Rules.Internal - Hakyll.Core.Runtime - Hakyll.Core.Store - Hakyll.Core.UnixFilter - Hakyll.Core.Util.File - Hakyll.Core.Util.String - Hakyll.Core.Writable - Hakyll.Main - Hakyll.Web.CompressCss - Hakyll.Web.Feed - Hakyll.Web.Html - Hakyll.Web.Html.RelativizeUrls - Hakyll.Web.Paginate - Hakyll.Web.Redirect - Hakyll.Web.Tags - Hakyll.Web.Template - Hakyll.Web.Template.Context - Hakyll.Web.Template.Internal - Hakyll.Web.Template.Internal.Element - Hakyll.Web.Template.Internal.Trim - Hakyll.Web.Template.List - hs-source-dirs: lib - other-modules: - Data.List.Extended - Data.Yaml.Extended - Hakyll.Check - Hakyll.Core.Compiler.Require - Hakyll.Core.Identifier.Pattern.Internal - Hakyll.Core.Item.SomeItem - Hakyll.Core.Provider.Internal - Hakyll.Core.Provider.MetadataCache - Hakyll.Core.Util.Parser - Paths_hakyll - default-language: Haskell2010 - ghc-options: -Wall - build-depends: - base >=4.8 && <5, - binary >=0.5, - blaze-html >=0.5, - blaze-markup >=0.5.1, - bytestring >=0.9, - containers >=0.3, - cryptonite >=0.25, - data-default >=0.4, - deepseq >=1.3, - directory >=1.0, - file-embed >=0.0.10.1, - filepath >=1.0, - lrucache >=1.1.1, - memory >=0.14.18, - mtl >=1, - network-uri >=2.6, - optparse-applicative >=0.12, - parsec >=3.0, - process >=1.6, - random >=1.0, - regex-tdfa >=1.1, - resourcet >=1.1, - scientific >=0.3.4, - tagsoup >=0.13.1, - template-haskell >=2.14, - text >=0.11, - time >=1.8, - time-locale-compat >=0.1, - unordered-containers >=0.2, - vector >=0.11, - yaml >=0.8.11 - - if flag(previewserver) - cpp-options: -DPREVIEW_SERVER - other-modules: - Hakyll.Preview.Poll - Hakyll.Preview.Server - build-depends: - wai >=3.2, - warp >=3.2, - wai-app-static >=3.1, - http-types >=0.9, - fsnotify >=0.2 - - if flag(watchserver) - cpp-options: -DWATCH_SERVER - other-modules: - Hakyll.Preview.Poll - build-depends: - fsnotify >=0.2 - - if flag(checkexternal) - cpp-options: -DCHECK_EXTERNAL - build-depends: - http-conduit >=2.2, - http-types >=0.7 - - if flag(usepandoc) - exposed-modules: - Hakyll.Web.Pandoc - Hakyll.Web.Pandoc.Biblio - Hakyll.Web.Pandoc.FileType - cpp-options: -DUSE_PANDOC - other-modules: - Hakyll.Web.Pandoc.Binary - build-depends: - pandoc >=2.0.5, - pandoc-citeproc >=0.14, - pandoc-types - -executable hakyll-init - main-is: Init.hs - hs-source-dirs: src - other-modules: - Paths_hakyll - default-language: Haskell2010 - ghc-options: -Wall -threaded - build-depends: - hakyll -any, - base >=4 && <5, - directory >=1.0, - filepath >=1.0 - -executable hakyll-website - main-is: site.hs - hs-source-dirs: web - default-language: Haskell2010 - ghc-options: -Wall -threaded - build-depends: - hakyll -any, - base >=4 && <5, - directory >=1.0, - filepath >=1.0, - pandoc >=2.0.5 - - if flag(buildwebsite) - else - buildable: False - -test-suite hakyll-tests - type: exitcode-stdio-1.0 - main-is: TestSuite.hs - hs-source-dirs: tests - other-modules: - Hakyll.Core.Dependencies.Tests - Hakyll.Core.Identifier.Tests - Hakyll.Core.Provider.Metadata.Tests - Hakyll.Core.Provider.Tests - Hakyll.Core.Routes.Tests - Hakyll.Core.Rules.Tests - Hakyll.Core.Runtime.Tests - Hakyll.Core.Store.Tests - Hakyll.Core.UnixFilter.Tests - Hakyll.Core.Util.String.Tests - Hakyll.Web.CompressCss.Tests - Hakyll.Web.Html.RelativizeUrls.Tests - Hakyll.Web.Html.Tests - Hakyll.Web.Tags.Tests - Hakyll.Web.Template.Context.Tests - Hakyll.Web.Template.Tests - TestSuite.Util - default-language: Haskell2010 - ghc-options: -Wall -threaded - build-depends: - hakyll -any, - QuickCheck >=2.8, - tasty >=0.11, - tasty-hunit >=0.9, - tasty-quickcheck >=0.8, - base >=4.8, - bytestring >=0.9, - containers >=0.3, - filepath >=1.0, - text >=0.11, - unordered-containers >=0.2, - yaml >=0.8.11 - - if flag(previewserver) - cpp-options: -DPREVIEW_SERVER - - if flag(watchserver) - cpp-options: -DWATCH_SERVER - - if flag(checkexternal) - cpp-options: -DCHECK_EXTERNAL - - if flag(usepandoc) - cpp-options: -DUSE_PANDOC - other-modules: - Hakyll.Web.Pandoc.FileType.Tests +Name: hakyll +Version: 4.14.0.0 + +Synopsis: A static website compiler library +Description: + Hakyll is a static website compiler library. It provides you with the tools to + create a simple or advanced static website using a Haskell DSL and formats + such as markdown or RST. You can find more information, including a tutorial, + on the website: + + . + + * <http://jaspervdj.be/hakyll> + + . + + If you seek assistance, there's: + + . + + * A google group: <http://groups.google.com/group/hakyll> + + . + + * An IRC channel, @#hakyll@ on irc.libera.chat (we *do not* have a channel on Freenode anymore) + + . + + Additionally, there's the Haddock documentation in the different modules, + meant as a reference. + +Author: Jasper Van der Jeugt <m@jaspervdj.be> +Maintainer: Jasper Van der Jeugt <m@jaspervdj.be> +Homepage: http://jaspervdj.be/hakyll +Bug-Reports: http://github.com/jaspervdj/Hakyll/issues +License: BSD3 +License-File: LICENSE +Category: Web + +Cabal-Version: >= 1.10 +Build-Type: Simple +Data-Dir: data + +Data-files: + example/posts/2015-11-28-carpe-diem.markdown + example/posts/2015-10-07-rosa-rosa-rosam.markdown + example/posts/2015-12-07-tu-quoque.markdown + example/posts/2015-08-12-spqr.markdown + example/site.hs + example/images/haskell-logo.png + example/templates/post-list.html + example/templates/default.html + example/templates/archive.html + example/templates/post.html + example/css/default.css + example/index.html + example/about.rst + example/contact.markdown + +Extra-source-files: + CHANGELOG.md + tests/data/biblio/biblio01.golden + tests/data/biblio/chicago.csl + tests/data/biblio/default.html + tests/data/biblio/page.markdown + tests/data/biblio/refs.bib + tests/data/embed.html + tests/data/example.md + tests/data/example.md.metadata + tests/data/images/favicon.ico + tests/data/just-meta.html + tests/data/just-meta.html.out + tests/data/partial-helper.html + tests/data/partial.html + tests/data/partial.html.out + tests/data/posts/2010-08-26-birthday.md + tests/data/posts/2018-09-26.md + tests/data/posts/2019/05/10/tomorrow.md + tests/data/russian.md + tests/data/strip.html + tests/data/strip.html.out + tests/data/template.html + tests/data/template.html.out + data/templates/atom-item.xml + data/templates/atom.xml + data/templates/rss-item.xml + data/templates/rss.xml + +Source-Repository head + Type: git + Location: git://github.com/jaspervdj/hakyll.git + +Flag previewServer + Description: Include the preview server + Default: True + +Flag watchServer + Description: Include the watch server + Default: True + +Flag checkExternal + Description: Include external link checking + Default: True + +Flag buildWebsite + Description: Build the hakyll website + Default: False + Manual: True + +Flag usePandoc + Description: Include Pandoc support + Default: True + Manual: True + +Library + Ghc-Options: -Wall + Hs-Source-Dirs: lib + Default-language: Haskell2010 + + Exposed-Modules: + Hakyll + Hakyll.Commands + Hakyll.Core.Compiler + Hakyll.Core.Compiler.Internal + Hakyll.Core.Configuration + Hakyll.Core.Dependencies + Hakyll.Core.File + Hakyll.Core.Identifier + Hakyll.Core.Identifier.Pattern + Hakyll.Core.Item + Hakyll.Core.Logger + Hakyll.Core.Metadata + Hakyll.Core.Provider + Hakyll.Core.Provider.Metadata + Hakyll.Core.Routes + Hakyll.Core.Rules + Hakyll.Core.Rules.Internal + Hakyll.Core.Runtime + Hakyll.Core.Store + Hakyll.Core.UnixFilter + Hakyll.Core.Util.File + Hakyll.Core.Util.String + Hakyll.Core.Writable + Hakyll.Main + Hakyll.Web.CompressCss + Hakyll.Web.Feed + Hakyll.Web.Html + Hakyll.Web.Html.RelativizeUrls + Hakyll.Web.Paginate + Hakyll.Web.Redirect + Hakyll.Web.Tags + Hakyll.Web.Template + Hakyll.Web.Template.Context + Hakyll.Web.Template.Internal + Hakyll.Web.Template.Internal.Element + Hakyll.Web.Template.Internal.Trim + Hakyll.Web.Template.List + + Other-Modules: + Data.List.Extended + Data.Yaml.Extended + Hakyll.Check + Hakyll.Core.Compiler.Require + Hakyll.Core.Identifier.Pattern.Internal + Hakyll.Core.Item.SomeItem + Hakyll.Core.Provider.Internal + Hakyll.Core.Provider.MetadataCache + Hakyll.Core.Util.Parser + Paths_hakyll + + Build-Depends: + array >= 0.5 && < 1, + base >= 4.8 && < 5, + binary >= 0.5 && < 0.10, + blaze-html >= 0.5 && < 0.10, + blaze-markup >= 0.5.1 && < 0.9, + bytestring >= 0.9 && < 0.11, + containers >= 0.3 && < 0.7, + data-default >= 0.4 && < 0.8, + deepseq >= 1.3 && < 1.5, + directory >= 1.2.7.0 && < 1.4, + file-embed >= 0.0.10.1 && < 0.0.15, + filepath >= 1.0 && < 1.5, + hashable >= 1.0 && < 2, + lifted-async >= 0.10 && < 1, + lrucache >= 1.1.1 && < 1.3, + mtl >= 1 && < 2.3, + network-uri >= 2.6 && < 2.7, + optparse-applicative >= 0.12 && < 0.17, + parsec >= 3.0 && < 3.2, + process >= 1.6 && < 1.7, + random >= 1.0 && < 1.3, + regex-tdfa >= 1.1 && < 1.4, + resourcet >= 1.1 && < 1.3, + scientific >= 0.3.4 && < 0.4, + stm >= 2.3 && < 3, + tagsoup >= 0.13.1 && < 0.15, + template-haskell >= 2.14 && < 2.18, + text >= 0.11 && < 1.3, + time >= 1.8 && < 1.12, + time-locale-compat >= 0.1 && < 0.2, + unordered-containers >= 0.2 && < 0.3, + vector >= 0.11 && < 0.13, + yaml >= 0.8.11 && < 0.12 + + If flag(previewServer) + Build-depends: + wai >= 3.2 && < 3.3, + warp >= 3.2 && < 3.4, + wai-app-static >= 3.1 && < 3.2, + http-types >= 0.9 && < 0.13, + fsnotify >= 0.2 && < 0.4 + Cpp-options: + -DPREVIEW_SERVER + Other-modules: + Hakyll.Preview.Poll + Hakyll.Preview.Server + + If flag(watchServer) + Build-depends: + fsnotify >= 0.2 && < 0.4 + Cpp-options: + -DWATCH_SERVER + Other-modules: + Hakyll.Preview.Poll + + If flag(checkExternal) + Build-depends: + http-conduit >= 2.2 && < 2.4, + http-types >= 0.7 && < 0.13 + Cpp-options: + -DCHECK_EXTERNAL + + If flag(usePandoc) + Exposed-Modules: + Hakyll.Web.Pandoc + Hakyll.Web.Pandoc.Biblio + Hakyll.Web.Pandoc.FileType + Other-Modules: + Hakyll.Web.Pandoc.Binary + Build-Depends: + pandoc >= 2.11 && < 2.15 + Cpp-options: + -DUSE_PANDOC + +Test-suite hakyll-tests + Type: exitcode-stdio-1.0 + Hs-source-dirs: tests + Main-is: TestSuite.hs + Ghc-options: -Wall -threaded + Default-language: Haskell2010 + + Other-modules: + Hakyll.Core.Dependencies.Tests + Hakyll.Core.Identifier.Tests + Hakyll.Core.Provider.Metadata.Tests + Hakyll.Core.Provider.Tests + Hakyll.Core.Routes.Tests + Hakyll.Core.Rules.Tests + Hakyll.Core.Runtime.Tests + Hakyll.Core.Store.Tests + Hakyll.Core.UnixFilter.Tests + Hakyll.Core.Util.String.Tests + Hakyll.Web.CompressCss.Tests + Hakyll.Web.Html.RelativizeUrls.Tests + Hakyll.Web.Html.Tests + Hakyll.Web.Tags.Tests + Hakyll.Web.Template.Context.Tests + Hakyll.Web.Template.Tests + TestSuite.Util + + Build-Depends: + hakyll, + QuickCheck >= 2.8 && < 2.15, + tasty >= 0.11 && < 1.5, + tasty-golden >= 2.3 && < 2.4, + tasty-hunit >= 0.9 && < 0.11, + tasty-quickcheck >= 0.8 && < 0.11, + -- Copy pasted from hakyll dependencies: + base >= 4.8 && < 5, + bytestring >= 0.9 && < 0.11, + containers >= 0.3 && < 0.7, + filepath >= 1.0 && < 1.5, + text >= 0.11 && < 1.3, + unordered-containers >= 0.2 && < 0.3, + yaml >= 0.8.11 && < 0.12 + + If flag(previewServer) + Cpp-options: + -DPREVIEW_SERVER + + If flag(watchServer) + Cpp-options: + -DWATCH_SERVER + + If flag(checkExternal) + Cpp-options: + -DCHECK_EXTERNAL + + If flag(usePandoc) + Other-modules: + Hakyll.Web.Pandoc.Biblio.Tests + Hakyll.Web.Pandoc.FileType.Tests + Cpp-options: + -DUSE_PANDOC + +Executable hakyll-init + Main-is: Init.hs + Ghc-options: -Wall -threaded + Hs-source-dirs: src + Default-language: Haskell2010 + + Other-modules: + Paths_hakyll + + Build-depends: + hakyll, + base >= 4 && < 5, + directory >= 1.0 && < 1.4, + filepath >= 1.0 && < 1.5 + +Executable hakyll-website + Main-is: site.hs + Ghc-options: -Wall -threaded + Hs-source-dirs: web + Default-language: Haskell2010 + + If flag(buildWebsite) + Buildable: True + Else + Buildable: False + + Build-depends: + hakyll, + base >= 4 && < 5, + directory >= 1.0 && < 1.4, + filepath >= 1.0 && < 1.5, + pandoc >= 2.11 && < 2.15 diff --git a/lib/Hakyll/Core/Dependencies.hs b/lib/Hakyll/Core/Dependencies.hs index 4a51b9c..f9b8048 100644 --- a/lib/Hakyll/Core/Dependencies.hs +++ b/lib/Hakyll/Core/Dependencies.hs @@ -33,6 +33,7 @@ import Hakyll.Core.Identifier.Pattern data Dependency = PatternDependency Pattern (Set Identifier) | IdentifierDependency Identifier + | AlwaysOutOfDate deriving (Show, Typeable) @@ -40,9 +41,11 @@ data Dependency instance Binary Dependency where put (PatternDependency p is) = putWord8 0 >> put p >> put is put (IdentifierDependency i) = putWord8 1 >> put i + put AlwaysOutOfDate = putWord8 2 get = getWord8 >>= \t -> case t of 0 -> PatternDependency <$> get <*> get 1 -> IdentifierDependency <$> get + 2 -> pure AlwaysOutOfDate _ -> error "Data.Binary.get: Invalid Dependency" @@ -84,13 +87,30 @@ markOod id' = State.modify $ \s -> -------------------------------------------------------------------------------- -dependenciesFor :: Identifier -> DependencyM [Identifier] +-- | Collection of dependencies that should be checked to determine +-- if an identifier needs rebuilding. +data Dependencies + = DependsOn [Identifier] + | MustRebuild + deriving (Show) + +instance Semigroup Dependencies where + DependsOn ids <> DependsOn moreIds = DependsOn (ids <> moreIds) + MustRebuild <> _ = MustRebuild + _ <> MustRebuild = MustRebuild + +instance Monoid Dependencies where + mempty = DependsOn [] + +-------------------------------------------------------------------------------- +dependenciesFor :: Identifier -> DependencyM Dependencies dependenciesFor id' = do facts <- dependencyFacts <$> State.get - return $ concatMap dependenciesFor' $ fromMaybe [] $ M.lookup id' facts + return $ foldMap dependenciesFor' $ fromMaybe [] $ M.lookup id' facts where - dependenciesFor' (IdentifierDependency i) = [i] - dependenciesFor' (PatternDependency _ is) = S.toList is + dependenciesFor' (IdentifierDependency i) = DependsOn [i] + dependenciesFor' (PatternDependency _ is) = DependsOn $ S.toList is + dependenciesFor' AlwaysOutOfDate = MustRebuild -------------------------------------------------------------------------------- @@ -113,6 +133,7 @@ checkChangedPatterns = do {dependencyFacts = M.insert id' deps' $ dependencyFacts s} where go _ ds (IdentifierDependency i) = return $ IdentifierDependency i : ds + go _ ds AlwaysOutOfDate = return $ AlwaysOutOfDate : ds go id' ds (PatternDependency p ls) = do universe <- ask let ls' = S.fromList $ filterMatches p universe @@ -136,11 +157,17 @@ bruteForce = do check (todo, changed) id' = do deps <- dependenciesFor id' - ood <- dependencyOod <$> State.get - case find (`S.member` ood) deps of - Nothing -> return (id' : todo, changed) - Just d -> do - tell [show id' ++ " is out-of-date because " ++ - show d ++ " is out-of-date"] - markOod id' - return (todo, True) + case deps of + DependsOn depList -> do + ood <- dependencyOod <$> State.get + case find (`S.member` ood) depList of + Nothing -> return (id' : todo, changed) + Just d -> do + tell [show id' ++ " is out-of-date because " ++ + show d ++ " is out-of-date"] + markOod id' + return (todo, True) + MustRebuild -> do + tell [show id' ++ " will be forcibly rebuilt"] + markOod id' + return (todo, True) diff --git a/lib/Hakyll/Core/Item.hs b/lib/Hakyll/Core/Item.hs index e05df42..af15b94 100644 --- a/lib/Hakyll/Core/Item.hs +++ b/lib/Hakyll/Core/Item.hs @@ -2,6 +2,7 @@ -- | An item is a combination of some content and its 'Identifier'. This way, we -- can still use the 'Identifier' to access metadata. {-# LANGUAGE DeriveDataTypeable #-} +{-# LANGUAGE DeriveTraversable #-} module Hakyll.Core.Item ( Item (..) , itemSetBody @@ -25,23 +26,7 @@ import Hakyll.Core.Identifier data Item a = Item { itemIdentifier :: Identifier , itemBody :: a - } deriving (Show, Typeable) - - --------------------------------------------------------------------------------- -instance Functor Item where - fmap f (Item i x) = Item i (f x) - - --------------------------------------------------------------------------------- -instance Foldable Item where - foldr f z (Item _ x) = f x z - - --------------------------------------------------------------------------------- -instance Traversable Item where - traverse f (Item i x) = Item i <$> f x - + } deriving (Show, Typeable, Functor, Foldable, Traversable) -------------------------------------------------------------------------------- instance Binary a => Binary (Item a) where diff --git a/lib/Hakyll/Core/Rules.hs b/lib/Hakyll/Core/Rules.hs index 41b9a73..695665a 100644 --- a/lib/Hakyll/Core/Rules.hs +++ b/lib/Hakyll/Core/Rules.hs @@ -29,6 +29,7 @@ module Hakyll.Core.Rules , preprocess , Dependency (..) , rulesExtraDependencies + , forceCompile ) where @@ -221,3 +222,11 @@ rulesExtraDependencies deps rules = | (i, c) <- rulesCompilers ruleSet ] } + + +-------------------------------------------------------------------------------- +-- | Force the item(s) to always be recompiled, whether or not the +-- dependencies are out of date. This can be useful if you are using +-- I/O to generate part (or all) of an item. +forceCompile :: Rules a -> Rules a +forceCompile = rulesExtraDependencies [AlwaysOutOfDate] diff --git a/lib/Hakyll/Core/Runtime.hs b/lib/Hakyll/Core/Runtime.hs index 68970cd..cc15b3e 100644 --- a/lib/Hakyll/Core/Runtime.hs +++ b/lib/Hakyll/Core/Runtime.hs @@ -5,19 +5,24 @@ module Hakyll.Core.Runtime -------------------------------------------------------------------------------- -import Control.Monad (unless) -import Control.Monad.Except (ExceptT, runExceptT, throwError) -import Control.Monad.Reader (ask) -import Control.Monad.RWS (RWST, runRWST) -import Control.Monad.State (get, modify) -import Control.Monad.Trans (liftIO) -import Data.List (intercalate) -import Data.Map (Map) -import qualified Data.Map as M -import Data.Set (Set) -import qualified Data.Set as S -import System.Exit (ExitCode (..)) -import System.FilePath ((</>)) +import Control.Concurrent.Async.Lifted (forConcurrently_) +import Control.Concurrent.STM (atomically, modifyTVar', readTVarIO, newTVarIO, TVar) +import Control.Monad (unless) +import Control.Monad.Except (ExceptT, runExceptT, throwError) +import Control.Monad.Reader (ask) +import Control.Monad.RWS (RWST, runRWST) +import Control.Monad.State (get) +import Control.Monad.Trans (liftIO) +import qualified Data.Array as A +import Data.Graph (Graph) +import qualified Data.Graph as G +import Data.List (intercalate) +import Data.Map (Map) +import qualified Data.Map as M +import Data.Set (Set) +import qualified Data.Set as S +import System.Exit (ExitCode (..)) +import System.FilePath ((</>)) -------------------------------------------------------------------------------- @@ -67,11 +72,13 @@ run config logger rules = do , runtimeRoutes = rulesRoutes ruleSet , runtimeUniverse = M.fromList compilers } - state = RuntimeState - { runtimeDone = S.empty - , runtimeSnapshots = S.empty - , runtimeTodo = M.empty - , runtimeFacts = oldFacts + + state <- newTVarIO $ RuntimeState + { runtimeDone = S.empty + , runtimeSnapshots = S.empty + , runtimeTodo = M.empty + , runtimeFacts = oldFacts + , runtimeDependencies = M.empty } -- Run the program and fetch the resulting state @@ -83,7 +90,8 @@ run config logger rules = do return (ExitFailure 1, ruleSet) Right (_, s, _) -> do - Store.set store factsKey $ runtimeFacts s + facts <- fmap runtimeFacts . liftIO . readTVarIO $ s + Store.set store factsKey facts Logger.debug logger "Removing tmp directory..." removeDirectory $ tmpDirectory config @@ -107,15 +115,30 @@ data RuntimeRead = RuntimeRead -------------------------------------------------------------------------------- data RuntimeState = RuntimeState - { runtimeDone :: Set Identifier - , runtimeSnapshots :: Set (Identifier, Snapshot) - , runtimeTodo :: Map Identifier (Compiler SomeItem) - , runtimeFacts :: DependencyFacts + { runtimeDone :: Set Identifier + , runtimeSnapshots :: Set (Identifier, Snapshot) + , runtimeTodo :: Map Identifier (Compiler SomeItem) + , runtimeFacts :: DependencyFacts + , runtimeDependencies :: Map Identifier (Set Identifier) } -------------------------------------------------------------------------------- -type Runtime a = RWST RuntimeRead () RuntimeState (ExceptT String IO) a +type Runtime a = RWST RuntimeRead () (TVar RuntimeState) (ExceptT String IO) a + + +-------------------------------------------------------------------------------- +-- Because compilation of rules often revolves around IO, +-- it is not possible to live in the STM monad and hence benefit from +-- its guarantees. +-- Be very careful when modifying the state +modifyRuntimeState :: (RuntimeState -> RuntimeState) -> Runtime () +modifyRuntimeState f = get >>= \s -> liftIO . atomically $ modifyTVar' s f + + +-------------------------------------------------------------------------------- +getRuntimeState :: Runtime RuntimeState +getRuntimeState = liftIO . readTVarIO =<< get -------------------------------------------------------------------------------- @@ -135,13 +158,15 @@ scheduleOutOfDate = do logger <- runtimeLogger <$> ask provider <- runtimeProvider <$> ask universe <- runtimeUniverse <$> ask - facts <- runtimeFacts <$> get - todo <- runtimeTodo <$> get let identifiers = M.keys universe modified = S.fromList $ flip filter identifiers $ resourceModified provider - + + state <- getRuntimeState + let facts = runtimeFacts state + todo = runtimeTodo state + let (ood, facts', msgs) = outOfDate identifiers modified facts todo' = M.filterWithKey (\id' _ -> id' `S.member` ood) universe @@ -150,7 +175,7 @@ scheduleOutOfDate = do mapM_ (Logger.debug logger) msgs -- Update facts and todo items - modify $ \s -> s + modifyRuntimeState $ \s -> s { runtimeDone = runtimeDone s `S.union` (S.fromList identifiers `S.difference` ood) , runtimeTodo = todo `M.union` todo' @@ -161,116 +186,138 @@ scheduleOutOfDate = do -------------------------------------------------------------------------------- pickAndChase :: Runtime () pickAndChase = do - todo <- runtimeTodo <$> get - case M.minViewWithKey todo of - Nothing -> return () - Just ((id', _), _) -> do - chase [] id' - pickAndChase + todo <- runtimeTodo <$> getRuntimeState + unless (null todo) $ do + checkForDependencyCycle + forConcurrently_ (M.keys todo) chase + pickAndChase -------------------------------------------------------------------------------- -chase :: [Identifier] -> Identifier -> Runtime () -chase trail id' - | id' `elem` trail = throwError $ "Hakyll.Core.Runtime.chase: " ++ - "Dependency cycle detected: " ++ intercalate " depends on " - (map show $ dropWhile (/= id') (reverse trail) ++ [id']) - | otherwise = do - logger <- runtimeLogger <$> ask - todo <- runtimeTodo <$> get - provider <- runtimeProvider <$> ask - universe <- runtimeUniverse <$> ask - routes <- runtimeRoutes <$> ask - store <- runtimeStore <$> ask - config <- runtimeConfiguration <$> ask - Logger.debug logger $ "Processing " ++ show id' - - let compiler = todo M.! id' - read' = CompilerRead - { compilerConfig = config - , compilerUnderlying = id' - , compilerProvider = provider - , compilerUniverse = M.keysSet universe - , compilerRoutes = routes - , compilerStore = store - , compilerLogger = logger +-- | Check for cyclic dependencies in the current state +checkForDependencyCycle :: Runtime () +checkForDependencyCycle = do + deps <- runtimeDependencies <$> getRuntimeState + let (depgraph, nodeFromVertex, _) = G.graphFromEdges [(k, k, S.toList dps) | (k, dps) <- M.toList deps] + dependencyCycles = map ((\(_, k, _) -> k) . nodeFromVertex) $ cycles depgraph + + unless (null dependencyCycles) $ do + throwError $ "Hakyll.Core.Runtime.pickAndChase: " ++ + "Dependency cycle detected: " ++ intercalate ", " (map show dependencyCycles) ++ + " are inter-dependent." + where + cycles :: Graph -> [G.Vertex] + cycles g = map fst . filter (uncurry $ reachableFromAny g) . A.assocs $ g + + reachableFromAny :: Graph -> G.Vertex -> [G.Vertex] -> Bool + reachableFromAny graph node = elem node . concatMap (G.reachable graph) + + +-------------------------------------------------------------------------------- +chase :: Identifier -> Runtime () +chase id' = do + logger <- runtimeLogger <$> ask + provider <- runtimeProvider <$> ask + universe <- runtimeUniverse <$> ask + routes <- runtimeRoutes <$> ask + store <- runtimeStore <$> ask + config <- runtimeConfiguration <$> ask + + state <- getRuntimeState + + Logger.debug logger $ "Processing " ++ show id' + + let compiler = (runtimeTodo state) M.! id' + read' = CompilerRead + { compilerConfig = config + , compilerUnderlying = id' + , compilerProvider = provider + , compilerUniverse = M.keysSet universe + , compilerRoutes = routes + , compilerStore = store + , compilerLogger = logger + } + + result <- liftIO $ runCompiler compiler read' + case result of + -- Rethrow error + CompilerError e -> throwError $ case compilerErrorMessages e of + [] -> "Compiler failed but no info given, try running with -v?" + es -> intercalate "; " es + + -- Signal that a snapshot was saved -> + CompilerSnapshot snapshot c -> do + -- Update info. The next 'chase' will pick us again at some + -- point so we can continue then. + modifyRuntimeState $ \s -> s + { runtimeSnapshots = S.insert (id', snapshot) (runtimeSnapshots s) + , runtimeTodo = M.insert id' c (runtimeTodo s) + } + + + -- Huge success + CompilerDone (SomeItem item) cwrite -> do + -- Print some info + let facts = compilerDependencies cwrite + cacheHits + | compilerCacheHits cwrite <= 0 = "updated" + | otherwise = "cached " + Logger.message logger $ cacheHits ++ " " ++ show id' + + -- Sanity check + unless (itemIdentifier item == id') $ throwError $ + "The compiler yielded an Item with Identifier " ++ + show (itemIdentifier item) ++ ", but we were expecting " ++ + "an Item with Identifier " ++ show id' ++ " " ++ + "(you probably want to call makeItem to solve this problem)" + + -- Write if necessary + (mroute, _) <- liftIO $ runRoutes routes provider id' + case mroute of + Nothing -> return () + Just route -> do + let path = destinationDirectory config </> route + liftIO $ makeDirectories path + liftIO $ write path item + Logger.debug logger $ "Routed to " ++ path + + -- Save! (For load) + liftIO $ save store item + + modifyRuntimeState $ \s -> s + { runtimeDone = S.insert id' (runtimeDone s) + , runtimeTodo = M.delete id' (runtimeTodo s) + , runtimeFacts = M.insert id' facts (runtimeFacts s) + } + + -- Try something else first + CompilerRequire dep c -> do + let (depId, depSnapshot) = dep + Logger.debug logger $ + "Compiler requirement found for: " ++ show id' ++ + ", requirement: " ++ show depId + + let done = runtimeDone state + snapshots = runtimeSnapshots state + deps = runtimeDependencies state + + -- Done if we either completed the entire item (runtimeDone) or + -- if we previously saved the snapshot (runtimeSnapshots). + let depDone = + depId `S.member` done || + (depId, depSnapshot) `S.member` snapshots + + let deps' = if depDone + then deps + else M.insertWith S.union id' (S.singleton depId) deps + + modifyRuntimeState $ \s -> s + { runtimeTodo = M.insert id' + (if depDone then c else compilerResult result) + (runtimeTodo s) + , runtimeDependencies = deps' } - result <- liftIO $ runCompiler compiler read' - case result of - -- Rethrow error - CompilerError e -> throwError $ case compilerErrorMessages e of - [] -> "Compiler failed but no info given, try running with -v?" - es -> intercalate "; " es - - -- Signal that a snapshot was saved -> - CompilerSnapshot snapshot c -> do - -- Update info. The next 'chase' will pick us again at some - -- point so we can continue then. - modify $ \s -> s - { runtimeSnapshots = - S.insert (id', snapshot) (runtimeSnapshots s) - , runtimeTodo = M.insert id' c (runtimeTodo s) - } - - -- Huge success - CompilerDone (SomeItem item) cwrite -> do - -- Print some info - let facts = compilerDependencies cwrite - cacheHits - | compilerCacheHits cwrite <= 0 = "updated" - | otherwise = "cached " - Logger.message logger $ cacheHits ++ " " ++ show id' - - -- Sanity check - unless (itemIdentifier item == id') $ throwError $ - "The compiler yielded an Item with Identifier " ++ - show (itemIdentifier item) ++ ", but we were expecting " ++ - "an Item with Identifier " ++ show id' ++ " " ++ - "(you probably want to call makeItem to solve this problem)" - - -- Write if necessary - (mroute, _) <- liftIO $ runRoutes routes provider id' - case mroute of - Nothing -> return () - Just route -> do - let path = destinationDirectory config </> route - liftIO $ makeDirectories path - liftIO $ write path item - Logger.debug logger $ "Routed to " ++ path - - -- Save! (For load) - liftIO $ save store item - - -- Update state - modify $ \s -> s - { runtimeDone = S.insert id' (runtimeDone s) - , runtimeTodo = M.delete id' (runtimeTodo s) - , runtimeFacts = M.insert id' facts (runtimeFacts s) - } - - -- Try something else first - CompilerRequire dep c -> do - -- Update the compiler so we don't execute it twice - let (depId, depSnapshot) = dep - done <- runtimeDone <$> get - snapshots <- runtimeSnapshots <$> get - - -- Done if we either completed the entire item (runtimeDone) or - -- if we previously saved the snapshot (runtimeSnapshots). - let depDone = - depId `S.member` done || - (depId, depSnapshot) `S.member` snapshots - - modify $ \s -> s - { runtimeTodo = M.insert id' - (if depDone then c else compilerResult result) - (runtimeTodo s) - } - - -- If the required item is already compiled, continue, or, start - -- chasing that - Logger.debug logger $ "Require " ++ show depId ++ - " (snapshot " ++ depSnapshot ++ "): " ++ - (if depDone then "OK" else "chasing") - if depDone then chase trail id' else chase (id' : trail) depId + Logger.debug logger $ "Require " ++ show depId ++ + " (snapshot " ++ depSnapshot ++ ") " +
\ No newline at end of file diff --git a/lib/Hakyll/Core/Store.hs b/lib/Hakyll/Core/Store.hs index bfcd191..da16c6f 100644 --- a/lib/Hakyll/Core/Store.hs +++ b/lib/Hakyll/Core/Store.hs @@ -16,20 +16,14 @@ module Hakyll.Core.Store -------------------------------------------------------------------------------- -import qualified Data.ByteArray as BA -import qualified Crypto.Hash as CH +import qualified Data.Hashable as DH import Data.Binary (Binary, decode, encodeFile) -import qualified Data.ByteString as B import qualified Data.ByteString.Lazy as BL import qualified Data.Cache.LRU.IO as Lru import Data.List (intercalate) import Data.Maybe (isJust) -import qualified Data.Text as T -import qualified Data.Text.Encoding as T import Data.Typeable (TypeRep, Typeable, cast, typeOf) -import Numeric (showHex) -import System.Directory (createDirectoryIfMissing) -import System.Directory (doesFileExist, removeFile) +import System.Directory (createDirectoryIfMissing, doesFileExist, removeFile) import System.FilePath ((</>)) import System.IO (IOMode (..), hClose, openFile) import System.IO.Error (catchIOError, ioeSetFileName, @@ -194,21 +188,4 @@ deleteFile = (`catchIOError` \_ -> return ()) . removeFile -------------------------------------------------------------------------------- -- | Mostly meant for internal usage hash :: [String] -> String -hash = toHex . B.unpack . hashMD5 . T.encodeUtf8 . T.pack . intercalate "/" - where - toHex [] = "" - toHex (x : xs) | x < 16 = '0' : showHex x (toHex xs) - | otherwise = showHex x (toHex xs) - - --------------------------------------------------------------------------------- --- | Hash by MD5 -hashMD5 :: B.ByteString -> B.ByteString -hashMD5 x = - let - digest :: CH.Digest CH.MD5 - digest = CH.hash x - bytes :: B.ByteString - bytes = BA.convert digest - in - bytes +hash = show . DH.hash . intercalate "/"
\ No newline at end of file diff --git a/lib/Hakyll/Core/Util/File.hs b/lib/Hakyll/Core/Util/File.hs index 9db6b11..02b8ece 100644 --- a/lib/Hakyll/Core/Util/File.hs +++ b/lib/Hakyll/Core/Util/File.hs @@ -1,3 +1,5 @@ +{-# LANGUAGE ScopedTypeVariables #-} +{-# LANGUAGE CPP #-} -------------------------------------------------------------------------------- -- | A module containing various file utility functions module Hakyll.Core.Util.File @@ -8,10 +10,12 @@ module Hakyll.Core.Util.File -------------------------------------------------------------------------------- +import Control.Concurrent (threadDelay) +import Control.Exception (SomeException, catch) import Control.Monad (filterM, forM, when) import System.Directory (createDirectoryIfMissing, doesDirectoryExist, getDirectoryContents, - removeDirectoryRecursive) + removeDirectoryRecursive, removePathForcibly) import System.FilePath (takeDirectory, (</>)) @@ -51,6 +55,30 @@ getRecursiveContents ignore top = go "" -------------------------------------------------------------------------------- removeDirectory :: FilePath -> IO () +#ifndef mingw32_HOST_OS removeDirectory fp = do e <- doesDirectoryExist fp when e $ removeDirectoryRecursive fp +#else +-- Deleting files on Windows is unreliable. If a file/directory is open by a program (e.g. antivirus), +-- then removing related directories *quickly* may fail with strange messages. +-- See here for discussions: +-- https://github.com/haskell/directory/issues/96 +-- https://github.com/haskell/win32/pull/129 +-- +-- The hacky solution is to retry deleting directories a few times, +-- with a delay, on Windows only. +removeDirectory = retryWithDelay 10 . removePathForcibly +#endif + + +-------------------------------------------------------------------------------- +-- | Retry an operation at most /n/ times (/n/ must be positive). +-- If the operation fails the /n/th time it will throw that final exception. +-- A delay of 100ms is introduced between every retry. +retryWithDelay :: Int -> IO a -> IO a +retryWithDelay i x + | i <= 0 = error "Hakyll.Core.Util.File.retry: retry count must be 1 or more" + | i == 1 = x + | otherwise = catch x $ \(_::SomeException) -> threadDelay 100 >> retryWithDelay (i-1) x + diff --git a/lib/Hakyll/Web/Html.hs b/lib/Hakyll/Web/Html.hs index 8cbfaa3..7aa3804 100644 --- a/lib/Hakyll/Web/Html.hs +++ b/lib/Hakyll/Web/Html.hs @@ -7,6 +7,7 @@ module Hakyll.Web.Html -- * Headers , demoteHeaders + , demoteHeadersBy -- * Url manipulation , getUrls @@ -50,13 +51,20 @@ withTagList f = renderTags' . f . parseTags' -------------------------------------------------------------------------------- -- | Map every @h1@ to an @h2@, @h2@ to @h3@, etc. demoteHeaders :: String -> String -demoteHeaders = withTags $ \tag -> case tag of +demoteHeaders = demoteHeadersBy 1 + +-------------------------------------------------------------------------------- +-- | Maps any @hN@ to an @hN+amount@ for any @amount > 0 && 1 <= N+amount <= 6@. +demoteHeadersBy :: Int -> String -> String +demoteHeadersBy amount + | amount < 1 = id + | otherwise = withTags $ \tag -> case tag of TS.TagOpen t a -> TS.TagOpen (demote t) a TS.TagClose t -> TS.TagClose (demote t) t -> t where demote t@['h', n] - | isDigit n = ['h', intToDigit (min 6 $ digitToInt n + 1)] + | isDigit n = ['h', intToDigit (min 6 $ digitToInt n + amount)] | otherwise = t demote t = t diff --git a/lib/Hakyll/Web/Pandoc.hs b/lib/Hakyll/Web/Pandoc.hs index 5f04de4..372465b 100644 --- a/lib/Hakyll/Web/Pandoc.hs +++ b/lib/Hakyll/Web/Pandoc.hs @@ -8,6 +8,8 @@ module Hakyll.Web.Pandoc , writePandocWith , renderPandoc , renderPandocWith + , renderPandocWithTransform + , renderPandocWithTransformM -- * Derived compilers , pandocCompiler @@ -104,6 +106,32 @@ renderPandocWith ropt wopt item = -------------------------------------------------------------------------------- +-- | An extension of `renderPandocWith`, which allows you to specify a custom +-- Pandoc transformation on the input `Item`. +-- Useful if you want to do your own transformations before running +-- custom Pandoc transformations, e.g. using a `funcField` to transform raw content. +renderPandocWithTransform :: ReaderOptions -> WriterOptions + -> (Pandoc -> Pandoc) + -> Item String + -> Compiler (Item String) +renderPandocWithTransform ropt wopt f = + renderPandocWithTransformM ropt wopt (return . f) + + +-------------------------------------------------------------------------------- +-- | Similar to `renderPandocWithTransform`, but the Pandoc transformation is +-- monadic. This is useful when you want the pandoc +-- transformation to use the `Compiler` information such as routes, +-- metadata, etc. along with your own transformations beforehand. +renderPandocWithTransformM :: ReaderOptions -> WriterOptions + -> (Pandoc -> Compiler Pandoc) + -> Item String + -> Compiler (Item String) +renderPandocWithTransformM ropt wopt f i = + writePandocWith wopt <$> (traverse f =<< readPandocWith ropt i) + + +-------------------------------------------------------------------------------- -- | Read a page render using pandoc pandocCompiler :: Compiler (Item String) pandocCompiler = @@ -137,9 +165,8 @@ pandocCompilerWithTransform ropt wopt f = pandocCompilerWithTransformM :: ReaderOptions -> WriterOptions -> (Pandoc -> Compiler Pandoc) -> Compiler (Item String) -pandocCompilerWithTransformM ropt wopt f = - writePandocWith wopt <$> - (traverse f =<< readPandocWith ropt =<< getResourceBody) +pandocCompilerWithTransformM ropt wopt f = + getResourceBody >>= renderPandocWithTransformM ropt wopt f -------------------------------------------------------------------------------- diff --git a/lib/Hakyll/Web/Pandoc/Biblio.hs b/lib/Hakyll/Web/Pandoc/Biblio.hs index 5127d88..566c706 100644 --- a/lib/Hakyll/Web/Pandoc/Biblio.hs +++ b/lib/Hakyll/Web/Pandoc/Biblio.hs @@ -12,6 +12,7 @@ {-# LANGUAGE Arrows #-} {-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} +{-# LANGUAGE OverloadedStrings #-} module Hakyll.Web.Pandoc.Biblio ( CSL , cslCompiler @@ -23,33 +24,31 @@ module Hakyll.Web.Pandoc.Biblio -------------------------------------------------------------------------------- -import Control.Monad (liftM, replicateM) -import Data.Binary (Binary (..)) -import Data.Typeable (Typeable) +import Control.Monad (liftM) +import Data.Binary (Binary (..)) +import qualified Data.ByteString as B +import qualified Data.ByteString.Lazy as BL +import qualified Data.Map as Map +import qualified Data.Time as Time +import Data.Typeable (Typeable) import Hakyll.Core.Compiler import Hakyll.Core.Compiler.Internal import Hakyll.Core.Identifier import Hakyll.Core.Item -import Hakyll.Core.Provider import Hakyll.Core.Writable import Hakyll.Web.Pandoc -import Hakyll.Web.Pandoc.Binary () -import qualified Text.CSL as CSL -import Text.CSL.Pandoc (processCites) -import Text.Pandoc (Pandoc, ReaderOptions (..), - enableExtension, Extension (..)) +import Text.Pandoc (Extension (..), Pandoc, + ReaderOptions (..), + enableExtension) +import qualified Text.Pandoc as Pandoc +import qualified Text.Pandoc.Citeproc as Pandoc (processCitations) -------------------------------------------------------------------------------- -data CSL = CSL - deriving (Show, Typeable) +newtype CSL = CSL {unCSL :: B.ByteString} + deriving (Binary, Show, Typeable) --------------------------------------------------------------------------------- -instance Binary CSL where - put CSL = return () - get = return CSL - -------------------------------------------------------------------------------- instance Writable CSL where @@ -59,21 +58,12 @@ instance Writable CSL where -------------------------------------------------------------------------------- cslCompiler :: Compiler (Item CSL) -cslCompiler = makeItem CSL - - --------------------------------------------------------------------------------- -newtype Biblio = Biblio [CSL.Reference] - deriving (Show, Typeable) +cslCompiler = fmap (CSL . BL.toStrict) <$> getResourceLBS -------------------------------------------------------------------------------- -instance Binary Biblio where - -- Ugly. - get = do - len <- get - Biblio <$> replicateM len get - put (Biblio rs) = put (length rs) >> mapM_ put rs +newtype Biblio = Biblio {unBiblio :: B.ByteString} + deriving (Binary, Show, Typeable) -------------------------------------------------------------------------------- @@ -84,12 +74,7 @@ instance Writable Biblio where -------------------------------------------------------------------------------- biblioCompiler :: Compiler (Item Biblio) -biblioCompiler = do - filePath <- getResourceFilePath - makeItem =<< unsafeCompiler (Biblio <$> CSL.readBiblioFile idpred filePath) - where - -- This is a filter on citations. We include all citations. - idpred = const True +biblioCompiler = fmap (Biblio . BL.toStrict) <$> getResourceLBS -------------------------------------------------------------------------------- @@ -99,21 +84,45 @@ readPandocBiblio :: ReaderOptions -> (Item String) -> Compiler (Item Pandoc) readPandocBiblio ropt csl biblio item = do - -- Parse CSL file, if given - provider <- compilerProvider <$> compilerAsk - style <- unsafeCompiler $ - CSL.readCSLFile Nothing . (resourceFilePath provider) . itemIdentifier $ csl + -- It's not straightforward to use the Pandoc API as of 2.11 to deal with + -- citations, since it doesn't export many things in 'Text.Pandoc.Citeproc'. + -- The 'citeproc' package is also hard to use. + -- + -- So instead, we try treating Pandoc as a black box. Pandoc can read + -- specific csl and bilbio files based on metadata keys. + -- + -- So we load the CSL and Biblio files and pass them to Pandoc using the + -- ersatz filesystem. + Pandoc.Pandoc (Pandoc.Meta meta) blocks <- itemBody <$> + readPandocWith ropt item + + let cslFile = Pandoc.FileInfo zeroTime . unCSL $ itemBody csl + bibFile = Pandoc.FileInfo zeroTime . unBiblio $ itemBody biblio + addBiblioFiles = \st -> st + { Pandoc.stFiles = + Pandoc.insertInFileTree "_hakyll/style.csl" cslFile . + Pandoc.insertInFileTree "_hakyll/refs.bib" bibFile $ + Pandoc.stFiles st + } + biblioMeta = Pandoc.Meta . + Map.insert "csl" (Pandoc.MetaString "_hakyll/style.csl") . + Map.insert "bibliography" (Pandoc.MetaString "_hakyll/refs.bib") $ + meta + errOrPandoc = Pandoc.runPure $ do + Pandoc.modifyPureState addBiblioFiles + Pandoc.processCitations $ Pandoc.Pandoc biblioMeta blocks - -- We need to know the citation keys, add then *before* actually parsing the - -- actual page. If we don't do this, pandoc won't even consider them - -- citations! - let Biblio refs = itemBody biblio - pandoc <- itemBody <$> readPandocWith ropt item - let pandoc' = processCites style refs pandoc + pandoc <- case errOrPandoc of + Left e -> compilerThrow ["Error during processCitations: " ++ show e] + Right x -> return x - return $ fmap (const pandoc') item + return $ fmap (const pandoc) item + + where + zeroTime = Time.UTCTime (toEnum 0) 0 -------------------------------------------------------------------------------- +-- | Compiles a markdown file via Pandoc. Requires the .csl and .bib files to be known to the compiler via match statements. pandocBiblioCompiler :: String -> String -> Compiler (Item String) pandocBiblioCompiler cslFileName bibFileName = do csl <- load $ fromFilePath cslFileName diff --git a/lib/Hakyll/Web/Pandoc/Binary.hs b/lib/Hakyll/Web/Pandoc/Binary.hs index 033ca9a..3f7f4fb 100644 --- a/lib/Hakyll/Web/Pandoc/Binary.hs +++ b/lib/Hakyll/Web/Pandoc/Binary.hs @@ -1,21 +1,20 @@ {-# OPTIONS_GHC -fno-warn-orphans #-} -{-# LANGUAGE CPP #-} {-# LANGUAGE DeriveGeneric #-} module Hakyll.Web.Pandoc.Binary where -import Data.Binary (Binary (..)) +import Data.Binary (Binary (..)) -import qualified Text.CSL as CSL -import qualified Text.CSL.Reference as REF -import qualified Text.CSL.Style as STY -import Text.Pandoc.Definition +import Text.Pandoc -------------------------------------------------------------------------------- -- orphans instance Binary Alignment instance Binary Block -instance Binary CSL.Reference +instance Binary Caption +instance Binary Cell +instance Binary ColSpan +instance Binary ColWidth instance Binary Citation instance Binary CitationMode instance Binary Format @@ -24,25 +23,9 @@ instance Binary ListNumberDelim instance Binary ListNumberStyle instance Binary MathType instance Binary QuoteType -instance Binary REF.CLabel -instance Binary REF.CNum -instance Binary REF.Literal -instance Binary REF.RefDate -instance Binary REF.RefType -instance Binary REF.Season -instance Binary STY.Agent -instance Binary STY.Formatted - -#if MIN_VERSION_pandoc_types(1, 21, 0) -instance Binary Caption -instance Binary Cell -instance Binary ColSpan -instance Binary ColWidth instance Binary Row instance Binary RowHeadColumns instance Binary RowSpan instance Binary TableBody instance Binary TableFoot instance Binary TableHead -#endif - diff --git a/lib/Hakyll/Web/Tags.hs b/lib/Hakyll/Web/Tags.hs index aab5d34..ccf34a5 100644 --- a/lib/Hakyll/Web/Tags.hs +++ b/lib/Hakyll/Web/Tags.hs @@ -43,6 +43,7 @@ module Hakyll.Web.Tags ( Tags (..) , getTags + , getTagsByField , getCategory , buildTagsWith , buildTags @@ -105,11 +106,16 @@ data Tags = Tags -- | Obtain tags from a page in the default way: parse them from the @tags@ -- metadata field. This can either be a list or a comma-separated string. getTags :: MonadMetadata m => Identifier -> m [String] -getTags identifier = do +getTags = getTagsByField "tags" + +-- | Obtain tags from a page by name of the metadata field. These can be a list +-- or a comma-separated string +getTagsByField :: MonadMetadata m => String -> Identifier -> m [String] +getTagsByField fieldName identifier = do metadata <- getMetadata identifier return $ fromMaybe [] $ - (lookupStringList "tags" metadata) `mplus` - (map trim . splitAll "," <$> lookupString "tags" metadata) + (lookupStringList fieldName metadata) `mplus` + (map trim . splitAll "," <$> lookupString fieldName metadata) -------------------------------------------------------------------------------- diff --git a/lib/Hakyll/Web/Template/Context.hs b/lib/Hakyll/Web/Template/Context.hs index 9dd14ff..97f0930 100644 --- a/lib/Hakyll/Web/Template/Context.hs +++ b/lib/Hakyll/Web/Template/Context.hs @@ -60,8 +60,7 @@ import Data.List (intercalate, tails) import Data.Semigroup (Semigroup (..)) #endif import Data.Time.Clock (UTCTime (..)) -import Data.Time.Format (formatTime) -import qualified Data.Time.Format as TF +import Data.Time.Format (formatTime, parseTimeM) import Data.Time.Locale.Compat (TimeLocale, defaultTimeLocale) import Hakyll.Core.Compiler import Hakyll.Core.Compiler.Internal @@ -466,10 +465,3 @@ teaserFieldWithSeparator separator key snapshot = field key $ \item -> do missingField :: Context a missingField = Context $ \k _ _ -> noResult $ "Missing field '" ++ k ++ "' in context" - -parseTimeM :: Bool -> TimeLocale -> String -> String -> Maybe UTCTime -#if MIN_VERSION_time(1,5,0) -parseTimeM = TF.parseTimeM -#else -parseTimeM _ = TF.parseTime -#endif diff --git a/src/Init.hs b/src/Init.hs index 09f8ed7..c79a76e 100644 --- a/src/Init.hs +++ b/src/Init.hs @@ -11,7 +11,8 @@ import Data.Char (isAlphaNum, isNumber) import Data.List (foldl', intercalate, isPrefixOf) import Data.Version (Version (..)) import System.Directory (canonicalizePath, copyFile, - doesFileExist) + doesFileExist, + setPermissions, getPermissions, writable) import System.Environment (getArgs, getProgName) import System.Exit (exitFailure) import System.FilePath (splitDirectories, (</>)) @@ -65,6 +66,10 @@ main = do putStrLn $ "Creating " ++ dst makeDirectories dst copyFile src dst + -- On some systems, the source folder may be readonly, + -- and copyFile will therefore create a readonly project... + p <- getPermissions dst + setPermissions dst (p {writable = True}) putStrLn $ "Creating " ++ cabalPath createCabal cabalPath name @@ -115,7 +120,7 @@ createCabal path name = , " main-is: site.hs" , " build-depends: base == 4.*" , " , hakyll == " ++ version' ++ ".*" - , " ghc-options: -threaded" + , " ghc-options: -threaded -rtsopts -with-rtsopts=-N" , " default-language: Haskell2010" ] where diff --git a/tests/Hakyll/Core/Runtime/Tests.hs b/tests/Hakyll/Core/Runtime/Tests.hs index 9c23162..615aaf2 100644 --- a/tests/Hakyll/Core/Runtime/Tests.hs +++ b/tests/Hakyll/Core/Runtime/Tests.hs @@ -8,6 +8,7 @@ module Hakyll.Core.Runtime.Tests -------------------------------------------------------------------------------- import qualified Data.ByteString as B import System.FilePath ((</>)) +import System.Exit (ExitCode (..)) import Test.Tasty (TestTree, testGroup) import Test.Tasty.HUnit (Assertion, (@?=)) @@ -22,7 +23,7 @@ import TestSuite.Util -------------------------------------------------------------------------------- tests :: TestTree tests = testGroup "Hakyll.Core.Runtime.Tests" $ - fromAssertions "run" [case01, case02] + fromAssertions "run" [case01, case02, case03] -------------------------------------------------------------------------------- @@ -94,3 +95,30 @@ case02 = do favicon @?= "Test" cleanTestEnv + + +-------------------------------------------------------------------------------- +-- Test that dependency cycles are correctly identified +case03 :: Assertion +case03 = do + logger <- Logger.new Logger.Error + (ec, _) <- run testConfiguration logger $ do + + create ["partial.html.out1"] $ do + route idRoute + compile $ do + example <- loadSnapshotBody "partial.html.out2" "raw" + makeItem example + >>= loadAndApplyTemplate "partial.html" defaultContext + + create ["partial.html.out2"] $ do + route idRoute + compile $ do + example <- loadSnapshotBody "partial.html.out1" "raw" + makeItem example + >>= loadAndApplyTemplate "partial.html" defaultContext + + + ec @?= ExitFailure 1 + + cleanTestEnv diff --git a/tests/Hakyll/Web/Html/Tests.hs b/tests/Hakyll/Web/Html/Tests.hs index cd362f4..9ab10bc 100644 --- a/tests/Hakyll/Web/Html/Tests.hs +++ b/tests/Hakyll/Web/Html/Tests.hs @@ -20,7 +20,18 @@ tests :: TestTree tests = testGroup "Hakyll.Web.Html.Tests" $ concat [ fromAssertions "demoteHeaders" [ "<h2>A h1 title</h2>" @=? - demoteHeaders "<h1>A h1 title</h1>" + demoteHeaders "<h1>A h1 title</h1>" -- Assert single-step demotion + , "<h6>A h6 title</h6>" @=? + demoteHeaders "<h6>A h6 title</h6>" -- Assert maximum demotion is h6 + ] + + , fromAssertions "demoteHeadersBy" + [ "<h3>A h1 title</h3>" @=? + demoteHeadersBy 2 "<h1>A h1 title</h1>" + , "<h6>A h5 title</h6>" @=? + demoteHeadersBy 2 "<h5>A h5 title</h5>" -- Assert that h6 is the lowest possible demoted header. + , "<h4>A h4 title</h4>" @=? + demoteHeadersBy 0 "<h4>A h4 title</h4>" -- Assert that a demotion of @N < 1@ is a no-op. ] , fromAssertions "withUrls" diff --git a/tests/Hakyll/Web/Pandoc/Biblio/Tests.hs b/tests/Hakyll/Web/Pandoc/Biblio/Tests.hs new file mode 100644 index 0000000..fb98f08 --- /dev/null +++ b/tests/Hakyll/Web/Pandoc/Biblio/Tests.hs @@ -0,0 +1,66 @@ +-------------------------------------------------------------------------------- +{-# LANGUAGE OverloadedStrings #-} +module Hakyll.Web.Pandoc.Biblio.Tests + ( tests + ) where + + +-------------------------------------------------------------------------------- +import System.FilePath ((</>)) +import Test.Tasty (TestTree, testGroup) +import Test.Tasty.Golden (goldenVsString) +import qualified Data.ByteString as B +import qualified Data.ByteString.Lazy as LBS + + +-------------------------------------------------------------------------------- +import Hakyll +import Hakyll.Core.Runtime +import Hakyll.Web.Pandoc.Biblio +import qualified Hakyll.Core.Logger as Logger +import TestSuite.Util + + +-------------------------------------------------------------------------------- +tests :: TestTree +tests = testGroup "Hakyll.Web.Pandoc.Biblio.Tests" $ + [ goldenTest01 + ] + +-------------------------------------------------------------------------------- +goldenTestsDataDir :: FilePath +goldenTestsDataDir = "tests/data/biblio" + +-------------------------------------------------------------------------------- +goldenTest01 :: TestTree +goldenTest01 = + goldenVsString + "biblio01" + (goldenTestsDataDir </> "biblio01.golden") + (do + -- Code lifted from https://github.com/jaspervdj/hakyll-citeproc-example. + logger <- Logger.new Logger.Error + let config = testConfiguration { providerDirectory = goldenTestsDataDir } + _ <- run config logger $ do + let myPandocBiblioCompiler = do + csl <- load "chicago.csl" + bib <- load "refs.bib" + getResourceBody >>= + readPandocBiblio defaultHakyllReaderOptions csl bib >>= + return . writePandoc + + match "default.html" $ compile templateCompiler + match "chicago.csl" $ compile cslCompiler + match "refs.bib" $ compile biblioCompiler + match "page.markdown" $ do + route $ setExtension "html" + compile $ + myPandocBiblioCompiler >>= + loadAndApplyTemplate "default.html" defaultContext + + output <- fmap LBS.fromStrict $ B.readFile $ + destinationDirectory testConfiguration </> "page.html" + + cleanTestEnv + + return output) diff --git a/tests/TestSuite.hs b/tests/TestSuite.hs index c3e32f8..7c18470 100644 --- a/tests/TestSuite.hs +++ b/tests/TestSuite.hs @@ -24,6 +24,7 @@ import qualified Hakyll.Web.CompressCss.Tests import qualified Hakyll.Web.Html.RelativizeUrls.Tests import qualified Hakyll.Web.Html.Tests #ifdef USE_PANDOC +import qualified Hakyll.Web.Pandoc.Biblio.Tests import qualified Hakyll.Web.Pandoc.FileType.Tests #endif import qualified Hakyll.Web.Template.Context.Tests @@ -48,6 +49,7 @@ main = defaultMain $ testGroup "Hakyll" , Hakyll.Web.Html.RelativizeUrls.Tests.tests , Hakyll.Web.Html.Tests.tests #ifdef USE_PANDOC + , Hakyll.Web.Pandoc.Biblio.Tests.tests , Hakyll.Web.Pandoc.FileType.Tests.tests #endif , Hakyll.Web.Tags.Tests.tests diff --git a/tests/data/biblio/biblio01.golden b/tests/data/biblio/biblio01.golden new file mode 100644 index 0000000..ace1e76 --- /dev/null +++ b/tests/data/biblio/biblio01.golden @@ -0,0 +1,16 @@ +<!doctype html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <title>This page cites a paper.</title> + </head> + <body> + <h1>This page cites a paper.</h1> + <p>I would like to cite one of my favourite papers <span class="citation" data-cites="meijer1991functional">(Meijer, Fokkinga, and Paterson 1991)</span> here.</p> +<div id="refs" class="references csl-bib-body hanging-indent" role="doc-bibliography"> +<div id="ref-meijer1991functional" class="csl-entry" role="doc-biblioentry"> +Meijer, Erik, Maarten Fokkinga, and Ross Paterson. 1991. <span>“Functional Programming with Bananas, Lenses, Envelopes and Barbed Wire.”</span> In <em>Conference on Functional Programming Languages and Computer Architecture</em>, 124–44. Springer. +</div> +</div> + </body> +</html> diff --git a/tests/data/biblio/chicago.csl b/tests/data/biblio/chicago.csl new file mode 100644 index 0000000..47d9eb8 --- /dev/null +++ b/tests/data/biblio/chicago.csl @@ -0,0 +1,648 @@ +<?xml version="1.0" encoding="utf-8"?> +<style xmlns="http://purl.org/net/xbiblio/csl" class="in-text" version="1.0" demote-non-dropping-particle="display-and-sort" page-range-format="chicago"> + <info> + <title>Chicago Manual of Style 17th edition (author-date)</title> + <id>http://www.zotero.org/styles/chicago-author-date</id> + <link href="http://www.zotero.org/styles/chicago-author-date" rel="self"/> + <link href="http://www.chicagomanualofstyle.org/tools_citationguide.html" rel="documentation"/> + <author> + <name>Julian Onions</name> + <email>julian.onions@gmail.com</email> + </author> + <contributor> + <name>Sebastian Karcher</name> + </contributor> + <contributor> + <name>Richard Karnesky</name> + <email>karnesky+zotero@gmail.com</email> + <uri>http://arc.nucapt.northwestern.edu/Richard_Karnesky</uri> + </contributor> + <contributor> + <name>Andrew Dunning</name> + <email>andrew.dunning@utoronto.ca</email> + <uri>https://orcid.org/0000-0003-0464-5036</uri> + </contributor> + <contributor> + <name>Matthew Roth</name> + <email>matthew.g.roth@yale.edu</email> + <uri> https://orcid.org/0000-0001-7902-6331</uri> + </contributor> + <category citation-format="author-date"/> + <category field="generic-base"/> + <summary>The author-date variant of the Chicago style</summary> + <updated>2018-01-24T12:00:00+00:00</updated> + <rights license="http://creativecommons.org/licenses/by-sa/3.0/">This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License</rights> + </info> + <locale xml:lang="en"> + <terms> + <term name="editor" form="verb-short">ed.</term> + <term name="container-author" form="verb">by</term> + <term name="translator" form="verb-short">trans.</term> + <term name="editortranslator" form="verb">edited and translated by</term> + <term name="translator" form="short">trans.</term> + </terms> + </locale> + <macro name="secondary-contributors"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="none"> + <group delimiter=". "> + <names variable="editor translator" delimiter=". "> + <label form="verb" text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + <names variable="director" delimiter=". "> + <label form="verb" text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </if> + </choose> + </macro> + <macro name="container-contributors"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <group prefix=", " delimiter=", "> + <names variable="container-author" delimiter=", "> + <label form="verb" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + <names variable="editor translator" delimiter=", "> + <label form="verb" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </if> + </choose> + </macro> + <macro name="editor"> + <names variable="editor"> + <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + </names> + </macro> + <macro name="translator"> + <names variable="translator"> + <name name-as-sort-order="first" and="text" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + </names> + </macro> + <macro name="recipient"> + <choose> + <if type="personal_communication"> + <choose> + <if variable="genre"> + <text variable="genre" text-case="capitalize-first"/> + </if> + <else> + <text term="letter" text-case="capitalize-first"/> + </else> + </choose> + </if> + </choose> + <names variable="recipient" delimiter=", "> + <label form="verb" prefix=" " text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </macro> + <macro name="substitute-title"> + <choose> + <if type="article-magazine article-newspaper review review-book" match="any"> + <text macro="container-title"/> + </if> + </choose> + </macro> + <macro name="contributors"> + <group delimiter=". "> + <names variable="author"> + <name and="text" name-as-sort-order="first" sort-separator=", " delimiter=", " delimiter-precedes-last="always"/> + <label form="short" prefix=", "/> + <substitute> + <names variable="editor"/> + <names variable="translator"/> + <names variable="director"/> + <text macro="substitute-title"/> + <text macro="title"/> + </substitute> + </names> + <text macro="recipient"/> + </group> + </macro> + <macro name="contributors-short"> + <names variable="author"> + <name form="short" and="text" delimiter=", " initialize-with=". "/> + <substitute> + <names variable="editor"/> + <names variable="translator"/> + <names variable="director"/> + <text macro="substitute-title"/> + <text macro="title"/> + </substitute> + </names> + </macro> + <macro name="interviewer"> + <names variable="interviewer" delimiter=", "> + <label form="verb" prefix=" " text-case="capitalize-first" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </macro> + <macro name="archive"> + <group delimiter=". "> + <text variable="archive_location" text-case="capitalize-first"/> + <text variable="archive"/> + <text variable="archive-place"/> + </group> + </macro> + <macro name="access"> + <group delimiter=". "> + <choose> + <if type="graphic report" match="any"> + <text macro="archive"/> + </if> + <else-if type="article-journal bill book chapter legal_case legislation motion_picture paper-conference" match="none"> + <text macro="archive"/> + </else-if> + </choose> + <choose> + <if type="webpage post-weblog" match="any"> + <date variable="issued" form="text"/> + </if> + </choose> + <choose> + <if variable="issued" match="none"> + <group delimiter=" "> + <text term="accessed" text-case="capitalize-first"/> + <date variable="accessed" form="text"/> + </group> + </if> + </choose> + <choose> + <if type="legal_case" match="none"> + <choose> + <if variable="DOI"> + <text variable="DOI" prefix="https://doi.org/"/> + </if> + <else> + <text variable="URL"/> + </else> + </choose> + </if> + </choose> + </group> + </macro> + <macro name="title"> + <choose> + <if variable="title" match="none"> + <choose> + <if type="personal_communication" match="none"> + <text variable="genre" text-case="capitalize-first"/> + </if> + </choose> + </if> + <else-if type="bill book graphic legislation motion_picture song" match="any"> + <text variable="title" text-case="title" font-style="italic"/> + <group prefix=" (" suffix=")" delimiter=" "> + <text term="version"/> + <text variable="version"/> + </group> + </else-if> + <else-if variable="reviewed-author"> + <choose> + <if variable="reviewed-title"> + <group delimiter=". "> + <text variable="title" text-case="title" quotes="true"/> + <group delimiter=", "> + <text variable="reviewed-title" text-case="title" font-style="italic" prefix="Review of "/> + <names variable="reviewed-author"> + <label form="verb-short" text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </group> + </if> + <else> + <group delimiter=", "> + <text variable="title" text-case="title" font-style="italic" prefix="Review of "/> + <names variable="reviewed-author"> + <label form="verb-short" text-case="lowercase" suffix=" "/> + <name and="text" delimiter=", "/> + </names> + </group> + </else> + </choose> + </else-if> + <else-if type="legal_case interview patent" match="any"> + <text variable="title"/> + </else-if> + <else> + <text variable="title" text-case="title" quotes="true"/> + </else> + </choose> + </macro> + <macro name="edition"> + <choose> + <if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <choose> + <if is-numeric="edition"> + <group delimiter=" " prefix=". "> + <number variable="edition" form="ordinal"/> + <text term="edition" form="short" strip-periods="true"/> + </group> + </if> + <else> + <text variable="edition" text-case="capitalize-first" prefix=". "/> + </else> + </choose> + </if> + <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if is-numeric="edition"> + <group delimiter=" " prefix=", "> + <number variable="edition" form="ordinal"/> + <text term="edition" form="short"/> + </group> + </if> + <else> + <text variable="edition" prefix=", "/> + </else> + </choose> + </else-if> + </choose> + </macro> + <macro name="locators"> + <choose> + <if type="article-journal"> + <choose> + <if variable="volume"> + <text variable="volume" prefix=" "/> + <group prefix=" (" suffix=")"> + <choose> + <if variable="issue"> + <text variable="issue"/> + </if> + <else> + <date variable="issued"> + <date-part name="month"/> + </date> + </else> + </choose> + </group> + </if> + <else-if variable="issue"> + <group delimiter=" " prefix=", "> + <text term="issue" form="short"/> + <text variable="issue"/> + <date variable="issued" prefix="(" suffix=")"> + <date-part name="month"/> + </date> + </group> + </else-if> + <else> + <date variable="issued" prefix=", "> + <date-part name="month"/> + </date> + </else> + </choose> + </if> + <else-if type="legal_case"> + <text variable="volume" prefix=", "/> + <text variable="container-title" prefix=" "/> + <text variable="page" prefix=" "/> + </else-if> + <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <group prefix=". " delimiter=". "> + <group> + <text term="volume" form="short" text-case="capitalize-first" suffix=" "/> + <number variable="volume" form="numeric"/> + </group> + <group> + <number variable="number-of-volumes" form="numeric"/> + <text term="volume" form="short" prefix=" " plural="true"/> + </group> + </group> + </else-if> + <else-if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if variable="page" match="none"> + <group prefix=". "> + <text term="volume" form="short" text-case="capitalize-first" suffix=" "/> + <number variable="volume" form="numeric"/> + </group> + </if> + </choose> + </else-if> + </choose> + </macro> + <macro name="locators-chapter"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <choose> + <if variable="page"> + <group prefix=", "> + <text variable="volume" suffix=":"/> + <text variable="page"/> + </group> + </if> + </choose> + </if> + </choose> + </macro> + <macro name="locators-article"> + <choose> + <if type="article-newspaper"> + <group prefix=", " delimiter=", "> + <group delimiter=" "> + <text variable="edition"/> + <text term="edition"/> + </group> + <group> + <text term="section" form="short" suffix=" "/> + <text variable="section"/> + </group> + </group> + </if> + <else-if type="article-journal"> + <choose> + <if variable="volume issue" match="any"> + <text variable="page" prefix=": "/> + </if> + <else> + <text variable="page" prefix=", "/> + </else> + </choose> + </else-if> + </choose> + </macro> + <macro name="point-locators"> + <choose> + <if variable="locator"> + <choose> + <if locator="page" match="none"> + <choose> + <if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <choose> + <if variable="volume"> + <group> + <text term="volume" form="short" suffix=" "/> + <number variable="volume" form="numeric"/> + <label variable="locator" form="short" prefix=", " suffix=" "/> + </group> + </if> + <else> + <label variable="locator" form="short" suffix=" "/> + </else> + </choose> + </if> + <else> + <label variable="locator" form="short" suffix=" "/> + </else> + </choose> + </if> + <else-if type="bill book graphic legal_case legislation motion_picture report song" match="any"> + <number variable="volume" form="numeric" suffix=":"/> + </else-if> + </choose> + <text variable="locator"/> + </if> + </choose> + </macro> + <macro name="container-prefix"> + <text term="in" text-case="capitalize-first"/> + </macro> + <macro name="container-title"> + <choose> + <if type="chapter entry-dictionary entry-encyclopedia paper-conference" match="any"> + <text macro="container-prefix" suffix=" "/> + </if> + </choose> + <choose> + <if type="webpage"> + <text variable="container-title" text-case="title"/> + </if> + <else-if type="legal_case" match="none"> + <group delimiter=" "> + <text variable="container-title" text-case="title" font-style="italic"/> + <choose> + <if type="post-weblog"> + <text value="(blog)"/> + </if> + </choose> + </group> + </else-if> + </choose> + </macro> + <macro name="publisher"> + <group delimiter=": "> + <text variable="publisher-place"/> + <text variable="publisher"/> + </group> + </macro> + <macro name="date"> + <choose> + <if variable="issued"> + <group delimiter=" "> + <date variable="original-date" form="text" date-parts="year" prefix="(" suffix=")"/> + <date variable="issued"> + <date-part name="year"/> + </date> + </group> + </if> + <else-if variable="status"> + <text variable="status" text-case="capitalize-first"/> + </else-if> + <else> + <text term="no date" form="short"/> + </else> + </choose> + </macro> + <macro name="date-in-text"> + <choose> + <if variable="issued"> + <group delimiter=" "> + <date variable="original-date" form="text" date-parts="year" prefix="[" suffix="]"/> + <date variable="issued"> + <date-part name="year"/> + </date> + </group> + </if> + <else-if variable="status"> + <text variable="status"/> + </else-if> + <else> + <text term="no date" form="short"/> + </else> + </choose> + </macro> + <macro name="day-month"> + <date variable="issued"> + <date-part name="month"/> + <date-part name="day" prefix=" "/> + </date> + </macro> + <macro name="collection-title"> + <choose> + <if match="none" type="article-journal"> + <choose> + <if match="none" is-numeric="collection-number"> + <group delimiter=", "> + <text variable="collection-title" text-case="title"/> + <text variable="collection-number"/> + </group> + </if> + <else> + <group delimiter=" "> + <text variable="collection-title" text-case="title"/> + <text variable="collection-number"/> + </group> + </else> + </choose> + </if> + </choose> + </macro> + <macro name="collection-title-journal"> + <choose> + <if type="article-journal"> + <group delimiter=" "> + <text variable="collection-title"/> + <text variable="collection-number"/> + </group> + </if> + </choose> + </macro> + <macro name="event"> + <group> + <text term="presented at" suffix=" "/> + <text variable="event"/> + </group> + </macro> + <macro name="description"> + <choose> + <if type="interview"> + <group delimiter=". "> + <text macro="interviewer"/> + <text variable="medium" text-case="capitalize-first"/> + </group> + </if> + <else-if type="patent"> + <group delimiter=" " prefix=". "> + <text variable="authority"/> + <text variable="number"/> + </group> + </else-if> + <else> + <text variable="medium" text-case="capitalize-first" prefix=". "/> + </else> + </choose> + <choose> + <if variable="title" match="none"/> + <else-if type="thesis personal_communication speech" match="any"/> + <else> + <group delimiter=" " prefix=". "> + <text variable="genre" text-case="capitalize-first"/> + <choose> + <if type="report"> + <text variable="number"/> + </if> + </choose> + </group> + </else> + </choose> + </macro> + <macro name="issue"> + <choose> + <if type="legal_case"> + <text variable="authority" prefix=". "/> + </if> + <else-if type="speech"> + <group prefix=". " delimiter=", "> + <group delimiter=" "> + <text variable="genre" text-case="capitalize-first"/> + <text macro="event"/> + </group> + <text variable="event-place"/> + <text macro="day-month"/> + </group> + </else-if> + <else-if type="article-newspaper article-magazine personal_communication" match="any"> + <date variable="issued" form="text" prefix=", "/> + </else-if> + <else-if type="patent"> + <group delimiter=", " prefix=", "> + <group delimiter=" "> + <!--Needs Localization--> + <text value="filed"/> + <date variable="submitted" form="text"/> + </group> + <group delimiter=" "> + <choose> + <if variable="issued submitted" match="all"> + <text term="and"/> + </if> + </choose> + <!--Needs Localization--> + <text value="issued"/> + <date variable="issued" form="text"/> + </group> + </group> + </else-if> + <else-if type="article-journal" match="any"/> + <else> + <group prefix=". " delimiter=", "> + <choose> + <if type="thesis"> + <text variable="genre" text-case="capitalize-first"/> + </if> + </choose> + <text macro="publisher"/> + </group> + </else> + </choose> + </macro> + <citation et-al-min="4" et-al-use-first="1" disambiguate-add-year-suffix="true" disambiguate-add-names="true" disambiguate-add-givenname="true" givenname-disambiguation-rule="primary-name" collapse="year"> + <layout prefix="(" suffix=")" delimiter="; "> + <group delimiter=", "> + <choose> + <if variable="issued accessed" match="any"> + <group delimiter=" "> + <text macro="contributors-short"/> + <text macro="date-in-text"/> + </group> + </if> + <!---comma before forthcoming and n.d.--> + <else> + <group delimiter=", "> + <text macro="contributors-short"/> + <text macro="date-in-text"/> + </group> + </else> + </choose> + <text macro="point-locators"/> + </group> + </layout> + </citation> + <bibliography hanging-indent="true" et-al-min="11" et-al-use-first="7" subsequent-author-substitute="———" entry-spacing="0"> + <sort> + <key macro="contributors"/> + <key variable="issued"/> + <key variable="title"/> + </sort> + <layout suffix="."> + <group delimiter=". "> + <text macro="contributors"/> + <text macro="date"/> + <text macro="title"/> + </group> + <text macro="description"/> + <text macro="secondary-contributors" prefix=". "/> + <text macro="container-title" prefix=". "/> + <text macro="container-contributors"/> + <text macro="edition"/> + <text macro="locators-chapter"/> + <text macro="collection-title-journal" prefix=", " suffix=", "/> + <text macro="locators"/> + <text macro="collection-title" prefix=". "/> + <text macro="issue"/> + <text macro="locators-article"/> + <text macro="access" prefix=". "/> + </layout> + </bibliography> +</style> diff --git a/tests/data/biblio/default.html b/tests/data/biblio/default.html new file mode 100644 index 0000000..42197e0 --- /dev/null +++ b/tests/data/biblio/default.html @@ -0,0 +1,11 @@ +<!doctype html> +<html lang="en"> + <head> + <meta charset="utf-8"> + <title>$title$</title> + </head> + <body> + <h1>$title$</h1> + $body$ + </body> +</html> diff --git a/tests/data/biblio/page.markdown b/tests/data/biblio/page.markdown new file mode 100644 index 0000000..5a99ac0 --- /dev/null +++ b/tests/data/biblio/page.markdown @@ -0,0 +1,5 @@ +--- +title: This page cites a paper. +--- + +I would like to cite one of my favourite papers [@meijer1991functional] here. diff --git a/tests/data/biblio/refs.bib b/tests/data/biblio/refs.bib new file mode 100644 index 0000000..e4cd89f --- /dev/null +++ b/tests/data/biblio/refs.bib @@ -0,0 +1,8 @@ +@inproceedings{meijer1991functional, + title={Functional programming with bananas, lenses, envelopes and barbed wire}, + author={Meijer, Erik and Fokkinga, Maarten and Paterson, Ross}, + booktitle={Conference on Functional Programming Languages and Computer Architecture}, + pages={124--144}, + year={1991}, + organization={Springer} +} diff --git a/web/examples.markdown b/web/examples.markdown index 3657ea0..c98937e 100644 --- a/web/examples.markdown +++ b/web/examples.markdown @@ -192,6 +192,12 @@ directly with the default Hakyll site. [source](https://github.com/rpearce/robertwpearce.com) - <https://blog.thjread.com/>, [source](https://github.com/thjread/thjread-blog) +- <https://xvw.github.io/>, + [source](https://github.com/xvw/planet) +- <https://jeancharles.quillet.org/>, + [source](https://github.com/jecaro/jeancharles.quillet) +- <https://nliu.net/>, + [source](https://github.com/dreamsmasher/site-haskell-source) ## Hakyll 3.X diff --git a/web/index.markdown b/web/index.markdown index bde5289..fd4c6ff 100644 --- a/web/index.markdown +++ b/web/index.markdown @@ -39,4 +39,5 @@ using [stack] by using `stack install hakyll`. Then, you can: - read the [tutorials](/tutorials.html); - mail the [google discussion group](http://groups.google.com/group/hakyll); - ask questions on the IRC channel: `#hakyll` on - [freenode](http://freenode.net/). + [irc.libera.chat](https://libera.chat/) (we *do not* have a channel on + Freenode anymore). diff --git a/web/templates/tutorial.html b/web/templates/tutorial.html index 54af861..a9c4a27 100644 --- a/web/templates/tutorial.html +++ b/web/templates/tutorial.html @@ -18,4 +18,5 @@ you have a github account, you can use the If you run into any problems, all questions are welcome in the above google group, or you could try the IRC channel, <code>#hakyll</code> on -<a href="http://freenode.net/">freenode</a>. +<a href="https://libera.chat/">irs.libera.chat</a> (we <em>do not</em> have +a channel on Freenode anymore). diff --git a/web/tutorials/01-installation.markdown b/web/tutorials/01-installation.markdown index 16e73c8..fb19026 100644 --- a/web/tutorials/01-installation.markdown +++ b/web/tutorials/01-installation.markdown @@ -21,7 +21,7 @@ Hakyll. 3. There are also some Linux distro packages: - - [Debian unstable](http://packages.debian.org/source/sid/haskell-hakyll) + - [Debian](https://packages.debian.org/source/stable/haskell-hakyll) - [Fedora](https://apps.fedoraproject.org/packages/ghc-hakyll) - [Nix]: `$ nix-env -iA nixos.haskellPackages.hakyll` @@ -51,11 +51,11 @@ use `cabal new-run site [command]`. You can build the site using: - my-site build + site build And preview (and build) it using: - my-site watch + site watch Using stack =========== diff --git a/web/tutorials/github-pages-tutorial.md b/web/tutorials/github-pages-tutorial.md index f79605c..2f6e82b 100644 --- a/web/tutorials/github-pages-tutorial.md +++ b/web/tutorials/github-pages-tutorial.md @@ -10,10 +10,7 @@ type: article Working with Hakyll on a GitHub Pages-hosted website is complicated slightly due to Hakyll outputting files to a ```_site``` subdirectory, our desire to have the source code as well as the compiled site stored in a single repository, and our desire to automate it. -This guide will walkthrough the creation and setup of a GitHub site that has two independent branches. - -1. ```master``` - This is where your site lives. It's what you see when you go to ```https://<your repo>.github.io```. This branch *needs* to be called master. -2. ```develop``` - This is where your website's source is. That's all your Haskell code, your posts and templates, etc, and it's where you do work from. This name was chosen arbitrarily and can be freely substituted for any name of your choosing. +This guide will walkthrough the creation and setup of a GitHub site that works on a single `master` branch. When you're finished, you will be able to, with one command, refresh your website's contents and send any changes to your GitHub Page. @@ -27,14 +24,15 @@ These instructions should be easy to adapt for any situation though. ## GitHub Setup -1. If required, create a new repository for your blog. -2. If required, create a ```master``` branch. -2. If applicable/desired, create/add to your repository any files that your site needs that will not be produced by your Hakyll project. For example, ```CNAME``` as outlined [here](https://help.github.com/articles/setting-up-your-pages-site-repository/). -3. Create a ```.gitignore``` file with at a minimum, the following entries: +1. If required, create a new GitHub repository for your blog. +2. If required, create a `master` branch. +3. in the Settings of your GitHub project define that the `/docs` folder from the `master` branch should be used as document-root of your site. + Please refer to the [documentation](https://docs.github.com/en/free-pro-team@latest/github/working-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site#choosing-a-publishing-source) + in case of problems. +4. Create a .gitignore file with at a minimum, the following entries: ``` _cache/ -_site/ .stack-work/ ``` @@ -47,10 +45,8 @@ _site/ 2. Create a ```.gitignore``` file in your blog's directory with at a minimum, the same directories listed as in the GitHub repository. 3. Use the following git commands to setup your local repository. -``` +```bash git init -# create new branch called develop and switch to it. -git checkout -b develop # track all the source files for our blog. git add . # make our first commit @@ -59,130 +55,75 @@ git commit -m "initial commit." git remote add origin <URL to your GitHub pages repository> ``` -## Deployment +### Modify site.hs -So everything's all setup and we're ready to deploy. +In order to make Hakyll generate the site into the `/docs` folder you'll have to edit the Hakyll Main module (`site.hs` if you use the stack template): -> **Note:** Performing the following commands from your ```develop``` branch is recommended since you will end up back in that branch at the end. +```haskell +config :: Configuration +config = defaultConfiguration + { destinationDirectory = "docs" + } -Temporarily save any uncommitted changes that may exist in the current branch. - -``` -git stash +main :: IO () +main = hakyllWith config $ do + ... ``` -Ensure we are in the correct branch. +## Deployment -``` -git checkout develop -``` +So everything’s all setup, and we’re ready to deploy. -We need to be able to run the executable that generates the website, so we need -to compile it first. If you are using `stack`, this can be done using: +We need to be able to run the executable that generates the website, so we need to compile it first. If you are using stack, this can be done using: -``` +```bash stack build ``` -Now get a clean build of our site. +Next we get a clean build of our site: -``` +```bash stack exec myblog clean stack exec myblog build ``` -Update the local list of remote branches to ensure we're able to checkout the branch we want in the next step. - -``` -git fetch --all -``` - -Switch to the `master` branch. - -> **Note:** Checking out this branch does not overwrite the files that Hakyll just produced because we have '_site' listed in both .gitignore files. - -``` -git checkout -b master --track origin/master -``` - -Next, copy the freshly made contents of '_site' over the old ones. +After this step you should see a folder `docs` under your projects root folder, which contains the generated Hakyll site. -> **Note:** Deleting a file from your site's source will not remove it from your `master` repository if it has already been published. An alternative to `cp` is discussed at the end of this guide. +Now we commit our changes: -``` -cp -a _site/. . -``` - -Commit our changes. - -``` +```bash git add -A git commit -m "Publish." ``` -And send them to GitHub. +And send them to GitHub: -``` +```bash git push origin master:master ``` -Final clean up and return to the original state. +That's all. -``` -git checkout develop -git branch -D master -git stash pop -``` +Within a few seconds your Hakyll site should be visible under your GitHub Pages URL! ## Putting it all together Below is a complete listing of all the commands used to automate deployment to Github Pages. A `deployCommand` can be set as part of Hakyll's configuration options. More information and an example is provided [here](https://jaspervdj.be/hakyll/reference/Hakyll-Core-Configuration.html). ``` -# Temporarily store uncommited changes -git stash - # Verify correct branch -git checkout develop +git checkout master # Build new files stack exec myblog clean stack exec myblog build -# Get previous files -git fetch --all -git checkout -b master --track origin/master - -# Overwrite existing files with new files -cp -a _site/. . - # Commit git add -A git commit -m "Publish." # Push git push origin master:master - -# Restoration -git checkout develop -git branch -D master -git stash pop -``` - -*And that's it.* - -## Removing old files with `rsync` - -Earlier it was mentioned a flaw is that deleted files will persist in the published site until deleted manually. This is easily overcome by using `rsync` instead of `cp`. - -``` -rsync -a --filter='P _site/' \ - --filter='P _cache/' \ - --filter='P .git/' \ - --filter='P .gitignore' \ - --filter='P .stack-work' \ - --delete-excluded \ - _site/ . ``` -The only drawback this approach has is the requirement that *every* file in your site "go through" Hakyll. Fortunately, in many cases this is not an issue. +*And that's it.*
\ No newline at end of file |