Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cabal-helper 1.0 #26

Merged
merged 33 commits into from
Nov 5, 2019
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
52940fb
Use cabal-helper 1.0
fendor Oct 3, 2019
26d0ddd
Update .gitmodules to use DanielG's cabal-helper
fendor Oct 11, 2019
a1bac07
Re-implement cabal-helper cradle
fendor Oct 13, 2019
294c401
Update hie-bios
fendor Oct 13, 2019
52b60ba
Fix builds for stack
fendor Oct 13, 2019
ae844a0
Change HaRe submodule to use different remote
fendor Oct 14, 2019
7def514
Update .gitmodules
fendor Oct 14, 2019
799bfd6
Fix multi-component support for cabal-helper cradle
fendor Oct 14, 2019
408b0b5
Add real error messages
fendor Oct 14, 2019
2fdcb3a
Add none-cradle if file does not belong to any package
fendor Oct 15, 2019
c41eed0
Fix cabal-helper multi-packages support
fendor Oct 15, 2019
2ffb17e
More Documentation
fendor Oct 15, 2019
b72e606
Refactor functions and add Documentation
fendor Oct 16, 2019
91a56b0
Improve comments
fendor Oct 25, 2019
b28e944
Upgrade stack version in circleci to 2.1.3
fendor Oct 25, 2019
c84b33f
.gitmodules, use https instead of ssh
fendor Oct 25, 2019
ed6d66b
Fix stack for ghc 8.6.5
fendor Oct 26, 2019
83c5090
Bump cabal-helper version to latest master a1c4a37
fendor Oct 26, 2019
cc40b6f
Implement perfect match for c-h-h cradle discovery
fendor Oct 28, 2019
7e7bd1d
Remove unused Language Pragma
fendor Oct 28, 2019
92add4e
Fix stripFilePath function
fendor Oct 29, 2019
c45714e
Remove comments from .gitmodules
fendor Oct 29, 2019
0517eaa
Implement the ancestors function
fendor Oct 29, 2019
97e6617
If not package can be found, return none-cradle
fendor Oct 29, 2019
b775f13
Prefer canonicalisePath over normalise
fendor Oct 29, 2019
b4f2326
Remove redundant check for stack installation
fendor Oct 29, 2019
9ceec1e
Move function relativeTo to the bottom of the file
fendor Oct 29, 2019
4309653
Move utility functions to the bottom of Cradle.hs
fendor Oct 31, 2019
9ee8156
Add exhautive documentation for Cabal-Helper-Helper implementation
fendor Oct 31, 2019
232e7d6
Update Documentation, e.g. fix typos and add explanations
fendor Nov 1, 2019
3ed7833
Fix typo in documentation of the project root discovery
fendor Nov 5, 2019
a19ff9c
Catch exceptions on initialisation and add explicit import list
fendor Nov 5, 2019
4a80ec3
Rework comments that do not make sense
fendor Nov 5, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ defaults: &defaults
- stack-cache-{{ .Environment.HIE_CACHE }}-{{ arch }}-{{ .Environment.CIRCLE_JOB }}-{{ checksum "stack-build.txt" }}
- stack-cache-{{ .Environment.HIE_CACHE }}-{{ arch }}-{{ .Environment.CIRCLE_JOB }}-{{ checksum "resolver.txt" }}

# - run:
# name: Stack upgrade
# command: stack upgrade
- run:
name: Stack upgrade
command: stack upgrade

- run:
name: Stack setup
Expand Down
9 changes: 5 additions & 4 deletions .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,17 @@
[submodule "submodules/cabal-helper"]
path = submodules/cabal-helper
# url = https://github.com/arbor/cabal-helper.git
url = https://github.com/alanz/cabal-helper.git
# url = https://github.com/DanielG/cabal-helper.git
# url = https://github.com/alanz/cabal-helper.git
url = https://github.com/DanielG/cabal-helper.git
# url = https://github.com/bubba/cabal-helper.git
fendor marked this conversation as resolved.
Show resolved Hide resolved

[submodule "submodules/ghc-mod"]
path = submodules/ghc-mod
# url = https://github.com/arbor/ghc-mod.git
url = https://github.com/alanz/ghc-mod.git
# url = https://github.com/bubba/ghc-mod.git
url = https://github.com/fendor/ghc-mod.git
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this dependency used for now?

Copy link
Collaborator Author

@fendor fendor Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ghc-project-types, which depends on c-h and hie-plugin-api depends on https://github.com/mpickering/haskell-ide-engine/blob/hie-bios/hie-plugin-api/Haskell/Ide/Engine/Ghc.hs#L301 which is exposed in ghc-project-types and HaRe uses this function.
We need this ghc-mod dep, because c-h has a minor change in the API. The diff in ghc-mod has like 3 changes, like, bumping c-h version, removing unknown type and adding instances for some type.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can eventually move ghc-project-types into another location if needed, possibly into hie itself.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I would love that! This would actually enable us to go to hackage, once c-h version 1.0 has been published!

#url = https://github.com/mpickering/ghc-mod.git

[submodule "hie-bios"]
path = hie-bios
url = https://github.com/mpickering/hie-bios.git
branch = multi-cradle
2 changes: 0 additions & 2 deletions app/MainHie.hs
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ import System.IO
import Haskell.Ide.Engine.Plugin.ApplyRefact
import Haskell.Ide.Engine.Plugin.Base
import Haskell.Ide.Engine.Plugin.Brittany
import Haskell.Ide.Engine.Plugin.Build
import Haskell.Ide.Engine.Plugin.Example2
import Haskell.Ide.Engine.Plugin.Bios
import Haskell.Ide.Engine.Plugin.HaRe
Expand All @@ -55,7 +54,6 @@ plugins includeExamples = pluginDescToIdePlugins allPlugins
[ applyRefactDescriptor "applyrefact"
, baseDescriptor "base"
, brittanyDescriptor "brittany"
, buildPluginDescriptor "build"
, haddockDescriptor "haddock"
, hareDescriptor "hare"
, hoogleDescriptor "hoogle"
Expand Down
3 changes: 1 addition & 2 deletions haskell-ide-engine.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ library
Haskell.Ide.Engine.Options
Haskell.Ide.Engine.Plugin.ApplyRefact
Haskell.Ide.Engine.Plugin.Brittany
Haskell.Ide.Engine.Plugin.Build
Haskell.Ide.Engine.Plugin.Example2
Haskell.Ide.Engine.Plugin.Floskell
Haskell.Ide.Engine.Plugin.Bios
Expand Down Expand Up @@ -58,7 +57,7 @@ library
, brittany
, bytestring
, Cabal
, cabal-helper >= 0.8.0.4
, cabal-helper >= 1.0 && < 1.1
, containers
, data-default
, directory
Expand Down
317 changes: 312 additions & 5 deletions hie-plugin-api/Haskell/Ide/Engine/Cradle.hs
Original file line number Diff line number Diff line change
@@ -1,6 +1,25 @@
module Haskell.Ide.Engine.Cradle (findLocalCradle) where
{-# LANGUAGE RecordWildCards #-}
{-# LANGUAGE NamedFieldPuns #-}
{-# LANGUAGE LambdaCase #-}
{-# LANGUAGE GADTs #-}

import HIE.Bios as BIOS
module Haskell.Ide.Engine.Cradle (findLocalCradle, isStackCradle) where

import HIE.Bios as BIOS
import HIE.Bios.Types as BIOS
import Haskell.Ide.Engine.MonadFunctions
import Distribution.Helper
import Distribution.Helper.Discover
import Data.Function ((&))
import qualified Data.List.NonEmpty as NonEmpty
import Data.List.NonEmpty (NonEmpty)
import System.FilePath
import qualified Data.Map as M
import Data.List (inits, sortOn, isPrefixOf, find, stripPrefix)
import Data.Maybe (listToMaybe, mapMaybe)
import Data.Ord (Down(..))
import Data.Foldable (toList)
import System.Exit

-- | Find the cradle that the given File belongs to.
--
Expand All @@ -14,8 +33,296 @@ import HIE.Bios as BIOS
-- of the project that may or may not be accurate.
findLocalCradle :: FilePath -> IO Cradle
findLocalCradle fp = do
-- Get the cabal directory from the cradle
cradleConf <- BIOS.findCradle fp
case cradleConf of
Just yaml -> BIOS.loadCradle yaml
Nothing -> BIOS.loadImplicitCradle fp
Just yaml -> fixCradle <$> BIOS.loadCradle yaml
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixCradle is only called when there is an explicit configuration but calls isStackCradle which checks for Cabal-Helper-Stack? Seems strange.

Copy link
Collaborator Author

@fendor fendor Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I wanted to reuse code.
isStackCradle can also be used by getProjectGhcVersion, and here, we know that if it is a stack cradle, it definitely has to be from the Bios cradle, which is the broken cradle. Do you think adding a separate function for fixCradle would make more sense?

Nothing -> cabalHelperCradle fp
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this then replace logic in hie-bios for implicitly finding a cradle with cabal-helper instead? Presumably cabal-helper handles this better then?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it does and in theory, yes it does.


-- | Check if the given Cradle is a stack cradle.
-- This might be used to determine the GHC version to use on the project.
-- If it is a stack-Cradle, we have to use `stack path --compiler-exe`
-- otherwise we may ask `ghc` directly what version it is.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really a good approach IMO, but probably an improvement over current behaviour.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, but we somehow have to know the ghc version to use on a project, otherwise HieWrapper can not work.

isStackCradle :: Cradle -> Bool
isStackCradle = (`elem` ["stack", "Cabal-Helper-Stack"])
. BIOS.actionName
. BIOS.cradleOptsProg

-- | Finds a Cabal v2-project, Cabal v1-project or a Stack project
-- relative to the given FilePath.
-- Cabal v2-project and Stack have priority over Cabal v1-project.
-- This entails that if a Cabal v1-project can be identified, it is
-- first checked whether there are Stack projects or Cabal v2-projects
-- before it is concluded that this is the project root.
-- Cabal v2-projects and Stack projects are equally important.
-- Due to the lack of user-input we have to guess which project it
-- should rather be.
-- This guessing has no guarantees and may change at any time.
findCabalHelperEntryPoint :: FilePath -> IO (Maybe (Ex ProjLoc))
findCabalHelperEntryPoint fp = do
projs <- concat <$> mapM findProjects subdirs
case filter (\p -> isCabalNewProject p || isStackProject p) projs of
(x:_) -> return $ Just x
[] -> case filter isCabalOldProject projs of
(x:_) -> return $ Just x
[] -> return Nothing

where
-- | Subdirectories of a given FilePath.
-- Directory closest to the FilePath `fp` is the head,
fendor marked this conversation as resolved.
Show resolved Hide resolved
-- followed by one directory taken away.
subdirs :: [FilePath]
subdirs = reverse . map joinPath . tail . inits
$ splitDirectories (takeDirectory fp)

isStackProject (Ex ProjLocStackYaml {}) = True
isStackProject _ = False

isCabalNewProject (Ex ProjLocV2Dir {}) = True
isCabalNewProject (Ex ProjLocV2File {}) = True
isCabalNewProject _ = False

isCabalOldProject (Ex ProjLocV1Dir {}) = True
isCabalOldProject (Ex ProjLocV1CabalFile {}) = True
isCabalOldProject _ = False

-- | Given a FilePath, find the Cradle the FilePath belongs to.
--
-- Finds the Cabal Package the FilePath is most likely a part of
-- and creates a cradle whose root directory is the directory
-- of the package the File belongs to.
cabalHelperCradle :: FilePath -> IO Cradle
cabalHelperCradle file = do
projM <- findCabalHelperEntryPoint file
case projM of
Nothing -> do
errorm $ "Could not find a Project for file: " ++ file
error $ "Could not find a Project for file: " ++ file
fendor marked this conversation as resolved.
Show resolved Hide resolved
Just (Ex proj) -> do
-- Find the root of the project based on project type.
let root = projectRootDir proj
-- Create a suffix for the cradle name.
-- Purpose is mainly for easier debugging.
let actionNameSuffix = projectSuffix proj
logm $ "Cabal-Helper dirs: " ++ show [root, file]
let dist_dir = getDefaultDistDir proj
env <- mkQueryEnv proj dist_dir
packages <- runQuery projectPackages env
-- Find the package the given file may belong to.
-- If it does not belong to any package, create a none-cradle.
-- We might want to find a cradle without actually loading anything.
-- Useful if we only want to determine a ghc version to use.
case packages `findPackageFor` file of
Nothing -> do
debugm $ "Could not find a package for the file: " ++ file
debugm
"This is perfectly fine if we only want to determine the GHC version."
return
Cradle { cradleRootDir = root
, cradleOptsProg =
CradleAction { actionName = "Cabal-Helper-"
++ actionNameSuffix
++ "-None"
, runCradle = \_ -> return CradleNone
}
}
Just realPackage -> do
debugm $ "Cabal-Helper cradle package: " ++ show realPackage
-- Field `pSourceDir` often has the form `<cwd>/./plugin`
fendor marked this conversation as resolved.
Show resolved Hide resolved
-- but we only want `<cwd>/plugin`
let normalisedPackageLocation = normalise $ pSourceDir realPackage
debugm
$ "Cabal-Helper normalisedPackageLocation: "
++ normalisedPackageLocation
return
Cradle { cradleRootDir = normalisedPackageLocation
, cradleOptsProg =
CradleAction { actionName =
"Cabal-Helper-" ++ actionNameSuffix
, runCradle = cabalHelperAction
env
realPackage
normalisedPackageLocation
}
}
where
-- | Cradle Action to query for the ComponentOptions that are needed
-- to load the given FilePath.
-- This Function is not supposed to throw any exceptions and use
-- 'CradleLoadResult' to indicate errors.
cabalHelperAction :: QueryEnv v -- ^ Query Env created by 'mkQueryEnv'
-- with the appropriate 'distdir'
-> Package v -- ^ Package this Cradle is part for.
-> FilePath -- ^ Root directory of the cradle
-- this action belongs to.
-> FilePath -- ^ FilePath to load, expected to be an absolute path.
-> IO (CradleLoadResult ComponentOptions)
cabalHelperAction env package root fp = do
-- Get all unit infos the given FilePath may belong to
let units = pUnits package
-- make the FilePath to load relative to the root of the cradle.
let relativeFp = makeRelative root fp
debugm $ "Relative Module FilePath: " ++ relativeFp
getComponent env (toList units) relativeFp
>>= \case
Just comp -> do
let fs = getFlags comp
let targets = getTargets comp relativeFp
let ghcOptions = fs ++ targets
debugm $ "Flags for \"" ++ fp ++ "\": " ++ show ghcOptions
debugm $ "Component Infos: " ++ show comp
return
$ CradleSuccess
ComponentOptions { componentOptions = ghcOptions
, componentDependencies = []
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No dependencies?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not right now, since HIE cant work with it anyways. In the end, this would be a reimplementation of the code in hie-bios.

}
Nothing -> return
$ CradleFail
$ CradleError
(ExitFailure 2)
("Could not obtain flags for " ++ fp)

-- | Get the component the given FilePath most likely belongs to.
-- Lazily ask units whether the given FilePath is part of their component.
fendor marked this conversation as resolved.
Show resolved Hide resolved
-- If a Module belongs to multiple components, it is not specified which
-- component will be loaded.
-- The given FilePath must be relative to the Root of the project
-- the given units belong to.
getComponent
:: QueryEnv pt -> [Unit pt] -> FilePath -> IO (Maybe ChComponentInfo)
getComponent _env [] _fp = return Nothing
getComponent env (unit:units) fp = do
ui <- runQuery (unitInfo unit) env
let components = M.elems (uiComponents ui)
debugm $ "Unit Info: " ++ show ui
case find (fp `partOfComponent`) components of
Nothing -> getComponent env units fp
comp {- Just component -} -> return comp

-- | Check whether the given FilePath is part of the Component.
-- A FilePath is part of the Component if and only if:
--
-- * One Component's 'ciSourceDirs' is a prefix of the FilePath
-- * The FilePath, after converted to a Module name,
-- is a in the Component's Targets, or the FilePath is
-- the executable in the component.
--
-- The latter is achieved by making the FilePath relative to the 'ciSourceDirs'
-- and then replacing Path separators with ".".
-- To check whether the given FilePath is the executable of the Component,
-- we have to check whether the FilePath, including 'ciSourceDirs',
-- is part of the targets in the Component.
partOfComponent ::
-- | FilePath relative to the package root.
FilePath ->
-- | Component to check whether the given FilePath is part of it.
ChComponentInfo ->
Bool
partOfComponent fp comp
| Just normFp <- normalisedFp fp (ciSourceDirs comp), normFp `inTargets` getTargets comp fp = True
| otherwise = False
where
-- | Assuming a FilePath "src/Lib/Lib.hs" and a ciSourceDirs ["src"]
-- into 'Just "Lib"'
-- >>> normalisedFp "src/Lib/Lib.hs" ["src"]
-- Just "Lib/Lib.hs"
--
-- >>> normalisedFp "src/Lib/Lib.hs" ["app"]
-- Nothing
normalisedFp file sourceDirs = listToMaybe
$ mapMaybe ((`stripPrefix` file) . addTrailingPathSeparator) sourceDirs

inTargets :: FilePath -> [String] -> Bool
inTargets modFp targets =
-- Change a FilePath of the Form "Haskell/IDE/Engine/Cradle.hs" -> "Haskell.IDE.Engine.Cradle"
let modName = map
(\c -> if isPathSeparator c
then '.'
else c)
(dropExtension modFp)
in any (`elem` targets) [modName, fp]

-- | Get the flags necessary to compile the given component.
getFlags :: ChComponentInfo -> [String]
getFlags = ciGhcOptions

-- | Get all Targets of a Component, since we want to load all components.
-- FilePath is needed for the special case that the Component is an Exe.
-- The Exe contains a Path to the Main which is relative to some entry
-- in 'ciSourceDirs'.
-- We monkey-patch this by supplying the FilePath we want to load,
-- which is part of this component, and select the 'ciSourceDir' we actually want.
-- See the Documentation of 'ciSourceDir' to why this contains multiple entries.
getTargets :: ChComponentInfo -> FilePath -> [String]
getTargets comp fp = case ciEntrypoints comp of
ChSetupEntrypoint {} -> []
ChLibEntrypoint { chExposedModules, chOtherModules }
-> map unChModuleName (chExposedModules ++ chOtherModules)
ChExeEntrypoint { chMainIs, chOtherModules }
-> [sourceDir </> chMainIs | Just sourceDir <- [sourceDirs]]
++ map unChModuleName chOtherModules
where
sourceDirs = find (`isFilePathPrefixOf` fp) (ciSourceDirs comp)

-- | For all packages in a project, find the project the given FilePath
-- belongs to most likely.
findPackageFor :: NonEmpty (Package pt) -> FilePath -> Maybe (Package pt)
findPackageFor packages fp = packages
& NonEmpty.toList
& sortOn (Down . pSourceDir)
& filter (\p -> pSourceDir p `isFilePathPrefixOf` fp)
& listToMaybe

-- | Helper function to make sure that both FilePaths are normalised.
-- Checks whether the first FilePath is a Prefix of the second FilePath.
-- Intended usage:
--
-- >>> isFilePathPrefixOf "./src/" "./src/File.hs"
-- True
--
-- >>> isFilePathPrefixOf "./src" "./src/File.hs"
-- True
--
-- >>> isFilePathPrefixOf "./src/././" "./src/File.hs"
-- True
--
-- >>> isFilePathPrefixOf "./src" "./src-dir/File.hs"
-- True -- This is not really intended.
isFilePathPrefixOf :: FilePath -> FilePath -> Bool
isFilePathPrefixOf dir fp = normalise dir `isPrefixOf` normalise fp
fendor marked this conversation as resolved.
Show resolved Hide resolved

projectRootDir :: ProjLoc qt -> FilePath
projectRootDir ProjLocV1CabalFile { plProjectDirV1 } = plProjectDirV1
projectRootDir ProjLocV1Dir { plProjectDirV1 } = plProjectDirV1
projectRootDir ProjLocV2File { plProjectDirV2 } = plProjectDirV2
projectRootDir ProjLocV2Dir { plProjectDirV2 } = plProjectDirV2
projectRootDir ProjLocStackYaml { plStackYaml } = takeDirectory plStackYaml

projectSuffix :: ProjLoc qt -> FilePath
projectSuffix ProjLocV1CabalFile {} = "Cabal-V1"
projectSuffix ProjLocV1Dir {} = "Cabal-V1-Dir"
projectSuffix ProjLocV2File {} = "Cabal-V2"
projectSuffix ProjLocV2Dir {} = "Cabal-V2-Dir"
projectSuffix ProjLocStackYaml {} = "Stack"

-- | The hie-bios stack cradle doesn't return the target as well, so add the
-- FilePath onto the end of the options to make sure at least one target
-- is returned.
fixCradle :: BIOS.Cradle -> BIOS.Cradle
fixCradle cradle =
-- Normally this would also succeed for the 'Cabal-Helper-Stack' cradle.
-- Make sure that the cradle is definitely the one created by "HIE.Bios.Cradle.loadCradle"
if isStackCradle cradle
then
-- We need a lens
cradle { BIOS.cradleOptsProg =
(BIOS.cradleOptsProg
cradle) { BIOS.runCradle = \fp' -> fmap (addOption fp')
<$> BIOS.runCradle
(BIOS.cradleOptsProg cradle)
fp'
}
}
else cradle
where
addOption fp (BIOS.ComponentOptions os ds) =
BIOS.ComponentOptions (os ++ [fp]) ds
Loading