Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kylegoetz udp #4844

Merged
merged 58 commits into from
May 10, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
af441db
add Network.UDP package
kylegoetz Mar 3, 2024
f6aac63
added UDP types and terms
kylegoetz Mar 3, 2024
14df922
delete commented-out code
kylegoetz Mar 29, 2024
f8e80a6
add UDP test
kylegoetz Mar 30, 2024
3679b0d
move UDP tests to correct spot
kylegoetz Mar 30, 2024
eb52fe7
removed commented-out code
kylegoetz Mar 30, 2024
35c4bdd
Merge branch 'kylegoetz-udp' of github.com:unisonweb/unison into kyle…
aryairani Mar 31, 2024
84b3a63
automatically run ormolu
aryairani Mar 31, 2024
6ac7a3a
disambiguation
aryairani Apr 2, 2024
e15757f
where do these white space changes keep coming from?
aryairani Apr 2, 2024
668c9b0
disambiguate transcripts
aryairani Apr 2, 2024
96a5be9
update transcript output
aryairani Apr 2, 2024
1be5cf2
Merge branch 'trunk' into kylegoetz-udp
aryairani Apr 2, 2024
0f62469
fixup testlib and add udp-tests.u to interpreter-tests.md
aryairani Apr 2, 2024
9341656
fixed UDP tests
kylegoetz Apr 2, 2024
178fa7d
Merge branch 'kylegoetz-udp' of https://github.com/unisonweb/unison i…
kylegoetz Apr 2, 2024
b202656
Merge branch 'kylegoetz-udp' of github.com:unisonweb/unison into kyle…
aryairani Apr 2, 2024
19be0a5
Merge remote-tracking branch 'origin/trunk' into kylegoetz-udp
aryairani Apr 5, 2024
fdf119a
update runtime_tests_version
aryairani Apr 5, 2024
db06066
whitespace
aryairani Apr 5, 2024
9192445
Merge remote-tracking branch 'origin/trunk' into kylegoetz-udp
aryairani Apr 5, 2024
eef5dda
Merge remote-tracking branch 'origin/trunk' into kylegoetz-udp
aryairani Apr 6, 2024
20b6ad4
note to self, build before pushing :-\
aryairani Apr 6, 2024
407ffdd
Merge branch 'trunk' into kylegoetz-udp
aryairani Apr 9, 2024
7845f40
add UDP for JIT
kylegoetz Apr 12, 2024
0bc69c3
Merge remote-tracking branch 'origin/trunk' into kylegoetz-udp
kylegoetz Apr 30, 2024
53cd4a5
primops updated
kylegoetz May 1, 2024
b24070c
sandbox
kylegoetz May 1, 2024
d6e37de
boot (defines new types)
kylegoetz May 1, 2024
612f01f
gitignore
kylegoetz May 1, 2024
e60a52c
fix udp defn in racket
kylegoetz May 1, 2024
7b2e520
temp JIT test script
kylegoetz May 1, 2024
2620820
add new types to boot.ss
kylegoetz May 1, 2024
bd31f1c
add UDPSocket, ListenSocket, ClientSockAddr
kylegoetz May 1, 2024
f7f306f
do not bind socket when creating client
kylegoetz May 1, 2024
5bd56cd
fix parens
kylegoetz May 1, 2024
6e85e52
.
kylegoetz May 1, 2024
b885790
cleanup
kylegoetz May 1, 2024
2ff6fa6
CSA.toText works
kylegoetz May 3, 2024
57a81a9
add builtin exporter to complement builtin-generated
kylegoetz May 3, 2024
3dddce6
fix builtin.rkt import
kylegoetz May 3, 2024
3b5d996
add UDP builtins to primops
kylegoetz May 3, 2024
56914af
tests passing
kylegoetz May 4, 2024
50b5e2a
ListenSocket.close
kylegoetz May 4, 2024
c62ee04
good
kylegoetz May 4, 2024
0729a96
extract handle-errors to shared file
kylegoetz May 6, 2024
0a62a78
refactor + tests pass
kylegoetz May 6, 2024
e2aa974
use my branch on Share for testing
kylegoetz May 6, 2024
9815b2f
remove UDPSocket/ListenSocket -> Socket extractions from Haskell
kylegoetz May 6, 2024
8e8b006
unused imports
kylegoetz May 6, 2024
4c9deb1
update transcript outputs
aryairani May 10, 2024
9da0a4f
Merge remote-tracking branch 'origin/trunk' into kylegoetz-udp
aryairani May 10, 2024
14c91e5
unmodify ci.yaml
aryairani May 10, 2024
b1a9030
revert .gitignore
aryairani May 10, 2024
d934078
whitespace
aryairani May 10, 2024
62b8bf2
whitespace
aryairani May 10, 2024
4890c2d
whitespace
aryairani May 10, 2024
4da7930
unmodify jit-tests.sh
aryairani May 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CREDITS.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@ These are listed in alphabetical order.
| [network-bsd-2.8.1.0](https://hackage.haskell.org/package/network-bsd-2.8.1.0) | [BSD3](https://hackage.haskell.org/package/network-bsd-2.8.1.0/src/LICENSE) |
| [network-info-0.2.0.10](https://hackage.haskell.org/package/network-info-0.2.0.10) | [BSD3](https://hackage.haskell.org/package/network-info-0.2.0.10/src/LICENSE) |
| [network-simple-0.4.5](https://hackage.haskell.org/package/network-simple-0.4.5) | [BSD3](https://hackage.haskell.org/package/network-simple-0.4.5/src/LICENSE) |

| [network-udp-0.0.0](https://hackage.haskell.org/package/network-udp-0.0.0) | [BSD3](https://hackage.haskell.org/package/network-udp-0.0.0/src/LICENSE) |

| [nonempty-containers-0.3.3.0](https://hackage.haskell.org/package/nonempty-containers-0.3.3.0) | [BSD3](https://hackage.haskell.org/package/nonempty-containers-0.3.3.0/src/LICENSE) |
| [nonempty-vector-0.2.0.2](https://hackage.haskell.org/package/nonempty-vector-0.2.0.2) | [BSD3](https://hackage.haskell.org/package/nonempty-vector-0.2.0.2/src/LICENSE) |
| [parallel-3.2.2.0](https://hackage.haskell.org/package/parallel-3.2.2.0) | [BSD3](https://hackage.haskell.org/package/parallel-3.2.2.0/src/LICENSE) |
Expand Down
2 changes: 2 additions & 0 deletions parser-typechecker/package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ dependencies:
- http-media
- http-types
- IntervalMap
- iproute
- lens
- lucid
- megaparsec
Expand All @@ -73,6 +74,7 @@ dependencies:
- natural-transformation
- network
- network-simple
- network-udp
- network-uri
- nonempty-containers
- open-browser
Expand Down
24 changes: 23 additions & 1 deletion parser-typechecker/src/Unison/Builtin.hs
Original file line number Diff line number Diff line change
Expand Up @@ -245,7 +245,10 @@ builtinTypesSrc =
B' "MutableArray" CT.Data,
B' "ImmutableByteArray" CT.Data,
B' "MutableByteArray" CT.Data,
B' "Char.Class" CT.Data
B' "Char.Class" CT.Data,
B' "UDPSocket" CT.Data,
B' "ListenSocket" CT.Data,
B' "ClientSockAddr" CT.Data
]

-- rename these to "builtin" later, when builtin means intrinsic as opposed to
Expand Down Expand Up @@ -810,6 +813,19 @@ ioBuiltins =
("IO.serverSocket.impl.v3", optionalt text --> text --> iof socket),
("IO.listen.impl.v3", socket --> iof unit),
("IO.clientSocket.impl.v3", text --> text --> iof socket),
("IO.UDP.clientSocket.impl.v1", text --> text --> iof udpSocket),
("IO.UDP.ClientSockAddr.toText.v1", udpClientSockAddr --> text),
("IO.UDP.UDPSocket.toText.impl.v1", udpSocket --> text),
("IO.UDP.UDPSocket.close.impl.v1", udpSocket --> iof unit),
("IO.UDP.UDPSocket.socket.impl.v1", udpSocket --> socket),
("IO.UDP.serverSocket.impl.v1", text --> text --> iof udpListenSocket),
("IO.UDP.ListenSocket.recvFrom.impl.v1", udpListenSocket --> iof (tuple [bytes, udpClientSockAddr])),
("IO.UDP.ListenSocket.sendTo.impl.v1", udpListenSocket --> bytes --> udpClientSockAddr --> iof unit),
("IO.UDP.ListenSocket.toText.impl.v1", udpListenSocket --> text),
("IO.UDP.ListenSocket.close.impl.v1", udpListenSocket --> iof unit),
("IO.UDP.ListenSocket.socket.impl.v1", udpListenSocket --> socket),
("IO.UDP.UDPSocket.recv.impl.v1", udpSocket --> iof bytes),
("IO.UDP.UDPSocket.send.impl.v1", udpSocket --> bytes --> iof unit),
("IO.closeSocket.impl.v3", socket --> iof unit),
("IO.socketPort.impl.v3", socket --> iof nat),
("IO.socketAccept.impl.v3", socket --> iof socket),
Expand Down Expand Up @@ -1050,6 +1066,12 @@ handle = Type.fileHandle ()
phandle = Type.processHandle ()
unit = DD.unitType ()


udpSocket, udpListenSocket, udpClientSockAddr :: Type
udpSocket = Type.udpSocket ()
udpListenSocket = Type.udpListenSocket ()
udpClientSockAddr = Type.udpClientSockAddr ()

tls, tlsClientConfig, tlsServerConfig, tlsSignedCert, tlsPrivateKey, tlsVersion, tlsCipher :: Type
tls = Type.ref () Type.tlsRef
tlsClientConfig = Type.ref () Type.tlsClientConfigRef
Expand Down
3 changes: 3 additions & 0 deletions parser-typechecker/src/Unison/KindInference/Generate.hs
Original file line number Diff line number Diff line change
Expand Up @@ -368,6 +368,9 @@ builtinConstraintTree =
flip Type.ref Type.filePathRef,
Type.threadId,
Type.socket,
Type.udpSocket,
Type.udpListenSocket,
Type.udpClientSockAddr,
Type.processHandle,
Type.ibytearrayType,
flip Type.ref Type.charClassRef,
Expand Down
119 changes: 118 additions & 1 deletion parser-typechecker/src/Unison/Runtime/Builtin.hs
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ import Data.IORef as SYS
readIORef,
writeIORef,
)
import Data.IP (IP)
import Data.Map qualified as Map
import Data.PEM (PEM, pemContent, pemParseLBS)
import Data.Set (insert)
Expand Down Expand Up @@ -81,9 +82,24 @@ import Network.Simple.TCP as SYS
import Network.Socket as SYS
( Socket,
accept,
socketPort,
socketPort, PortNumber,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this eventually won't pass ormolu.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the workflows a bit to be more flexible, we'll see what happens.

)
import Network.TLS as TLS
import Network.UDP as UDP
( UDPSocket (..),
ClientSockAddr,
ListenSocket (listenSocket),
clientSocket,
close,
recv,
recvFrom,
send,
sendTo,
serverSocket,
stop,
udpSocket,
)

import Network.TLS.Extra.Cipher as Cipher
import System.Clock (Clock (..), getTime, nsec, sec)
import System.Directory as SYS
Expand Down Expand Up @@ -1544,6 +1560,22 @@ outIoFailBool stack1 stack2 stack3 extra fail result =
)
]

outIoFailTup :: forall v . (Var v) => v -> v -> v -> v -> v -> v -> v -> v -> ANormal v
outIoFailTup stack1 stack2 stack3 stack4 stack5 extra fail result =
TMatch result . MatchSum $
mapFromList
[ failureCase stack1 stack2 stack3 extra fail,
( 1,
([BX, BX],
TAbss [stack1, stack2]
. TLetD stack3 BX (TCon Ty.unitRef 0 [])
. TLetD stack4 BX (TCon Ty.pairRef 0 [stack2, stack3])
. TLetD stack5 BX (TCon Ty.pairRef 0 [stack1, stack4])
$ right stack5
)
)
]

outIoFailG ::
(Var v) =>
v ->
Expand Down Expand Up @@ -1767,6 +1799,14 @@ boxToEFBox =
where
(arg, result, stack1, stack2, stack3, any, fail) = fresh

-- a -> Either Failure (b, c)
boxToEFTup :: ForeignOp
boxToEFTup =
inBx arg result $
outIoFailTup stack1 stack2 stack3 stack4 stack5 extra fail result
where
(arg, result, stack1, stack2, stack3, stack4, stack5, extra, fail) = fresh

-- a -> Either Failure (Maybe b)
boxToEFMBox :: ForeignOp
boxToEFMBox =
Expand Down Expand Up @@ -1858,6 +1898,14 @@ boxBoxToEF0 =
where
(arg1, arg2, result, stack1, stack2, stack3, fail, unit) = fresh

-- a -> b -> c -> Either Failure ()
boxBoxBoxToEF0 :: ForeignOp
boxBoxBoxToEF0 =
inBxBxBx arg1 arg2 arg3 result $
outIoFailUnit stack1 stack2 stack3 fail unit result
where
(arg1, arg2, arg3, result, stack1, stack2, stack3, fail, unit) = fresh

-- a -> Either Failure Nat
boxToEFNat :: ForeignOp
boxToEFNat =
Expand Down Expand Up @@ -2290,8 +2338,77 @@ mkForeignTlsE f = mkForeign $ \a -> fmap flatten (tryIO2 (tryIO1 (f a)))
flatten (Right (Right (Left e))) = Left e
flatten (Right (Right (Right a))) = Right a

declareUdpForeigns :: FDecl Symbol ()
declareUdpForeigns = do
declareForeign Tracked "IO.UDP.clientSocket.impl.v1" boxBoxToEFBox
. mkForeignIOF
$ \(host :: Util.Text.Text, port :: Util.Text.Text) ->
let hostStr = Util.Text.toString host
portStr = Util.Text.toString port
in UDP.clientSocket hostStr portStr True

declareForeign Tracked "IO.UDP.UDPSocket.recv.impl.v1" boxToEFBox
. mkForeignIOF
$ \(sock :: UDPSocket) -> Bytes.fromArray <$> UDP.recv sock

-- declareForeign Tracked "IO.UDP.UDPSocket.send.impl.v1" boxBoxToExn0
-- . mkForeign $ fmap .uncurry UDP.send

kylegoetz marked this conversation as resolved.
Show resolved Hide resolved
declareForeign Tracked "IO.UDP.UDPSocket.send.impl.v1" boxBoxToEF0
. mkForeignIOF
$ \(sock :: UDPSocket, bytes :: Bytes.Bytes) ->
UDP.send sock (Bytes.toArray bytes)

declareForeign Tracked "IO.UDP.UDPSocket.socket.impl.v1" boxToEFBox
. mkForeignIOF
$ \(socket:: UDPSocket) -> pure $ udpSocket socket

declareForeign Tracked "IO.UDP.UDPSocket.socket.impl.v1" boxDirect
. mkForeign $ pure . udpSocket

declareForeign Tracked "IO.UDP.UDPSocket.close.impl.v1" boxToEF0
. mkForeignIOF
$ \(sock :: UDPSocket) -> UDP.close sock

declareForeign Tracked "IO.UDP.ListenSocket.close.impl.v1" boxToEF0
. mkForeignIOF
$ \(sock :: ListenSocket) -> UDP.stop sock

declareForeign Tracked "IO.UDP.UDPSocket.toText.impl.v1" boxDirect
. mkForeign
$ \(sock :: UDPSocket) -> pure $ show sock

declareForeign Tracked "IO.UDP.serverSocket.impl.v1" boxBoxToEFBox
. mkForeignIOF
$ \(ip :: Util.Text.Text, port :: Util.Text.Text) ->
let maybeIp = readMaybe $ Util.Text.toString ip :: Maybe IP
maybePort = readMaybe $ Util.Text.toString port :: Maybe PortNumber
in case (maybeIp, maybePort) of
(Nothing, _) -> fail "Invalid IP Address"
(_, Nothing) -> fail "Invalid Port Number"
(Just ip, Just pt) -> UDP.serverSocket (ip, pt)

declareForeign Tracked "IO.UDP.ListenSocket.toText.impl.v1" boxDirect
. mkForeign
$ \(sock :: ListenSocket) -> pure $ show sock

declareForeign Tracked "IO.UDP.ListenSocket.recvFrom.impl.v1" boxToEFTup .
mkForeignIOF $ fmap (first Bytes.fromArray) <$> UDP.recvFrom

declareForeign Tracked "IO.UDP.ClientSockAddr.toText.v1" boxDirect
. mkForeign
$ \(sock :: ClientSockAddr) -> pure $ show sock

declareForeign Tracked "IO.UDP.ListenSocket.sendTo.impl.v1" boxBoxBoxToEF0 .
mkForeignIOF $ \(socket :: ListenSocket, bytes :: Bytes.Bytes, addr :: ClientSockAddr) ->
UDP.sendTo socket (Bytes.toArray bytes) addr

declareForeign Tracked "IO.UDP.ListenSocket.socket.impl.v1" boxDirect
. mkForeign $ pure . listenSocket

declareForeigns :: FDecl Symbol ()
declareForeigns = do
declareUdpForeigns
declareForeign Tracked "IO.openFile.impl.v3" boxIomrToEFBox $
mkForeignIOF $ \(fnameText :: Util.Text.Text, n :: Int) ->
let fname = Util.Text.toString fnameText
Expand Down
12 changes: 12 additions & 0 deletions parser-typechecker/src/Unison/Runtime/Foreign.hs
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import Data.Primitive (ByteArray, MutableArray, MutableByteArray)
import Data.Tagged (Tagged (..))
import Data.X509 qualified as X509
import Network.Socket (Socket)
import Network.UDP (ListenSocket, UDPSocket, ClientSockAddr)
import Network.TLS qualified as TLS (ClientParams, Context, ServerParams)
import System.Clock (TimeSpec)
import System.IO (Handle)
Expand Down Expand Up @@ -81,6 +82,10 @@ socketEq :: Socket -> Socket -> Bool
socketEq l r = l == r
{-# NOINLINE socketEq #-}

udpSocketEq :: UDPSocket -> UDPSocket -> Bool
udpSocketEq l r = l == r
{-# NOINLINE udpSocketEq #-}

refEq :: IORef () -> IORef () -> Bool
refEq l r = l == r
{-# NOINLINE refEq #-}
Expand Down Expand Up @@ -157,6 +162,7 @@ ref2eq r
-- Ditto
| r == Ty.tvarRef = Just $ promote tvarEq
| r == Ty.socketRef = Just $ promote socketEq
| r == Ty.udpSocketRef = Just $ promote udpSocketEq
| r == Ty.refRef = Just $ promote refEq
| r == Ty.threadIdRef = Just $ promote tidEq
| r == Ty.marrayRef = Just $ promote marrEq
Expand Down Expand Up @@ -230,6 +236,12 @@ instance BuiltinForeign Referent where foreignRef = Tagged Ty.termLinkRef

instance BuiltinForeign Socket where foreignRef = Tagged Ty.socketRef

instance BuiltinForeign ListenSocket where foreignRef = Tagged Ty.udpListenSocketRef

instance BuiltinForeign ClientSockAddr where foreignRef = Tagged Ty.udpClientSockAddrRef

instance BuiltinForeign UDPSocket where foreignRef = Tagged Ty.udpSocketRef

instance BuiltinForeign ThreadId where foreignRef = Tagged Ty.threadIdRef

instance BuiltinForeign TLS.ClientParams where foreignRef = Tagged Ty.tlsClientConfigRef
Expand Down
5 changes: 5 additions & 0 deletions parser-typechecker/src/Unison/Runtime/Foreign/Function.hs
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import Data.Time.Clock.POSIX (POSIXTime)
import Data.Word (Word16, Word32, Word64, Word8)
import GHC.IO.Exception (IOErrorType (..), IOException (..))
import Network.Socket (Socket)
import Network.UDP (UDPSocket)
import System.IO (BufferMode (..), Handle, IOMode, SeekMode)
import Unison.Builtin.Decls qualified as Ty
import Unison.Reference (Reference)
Expand Down Expand Up @@ -139,6 +140,10 @@ instance ForeignConvention Socket where
readForeign = readForeignBuiltin
writeForeign = writeForeignBuiltin

instance ForeignConvention UDPSocket where
readForeign = readForeignBuiltin
writeForeign = writeForeignBuiltin

instance ForeignConvention ThreadId where
readForeign = readForeignBuiltin
writeForeign = writeForeignBuiltin
Expand Down
4 changes: 4 additions & 0 deletions parser-typechecker/unison-parser-typechecker.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,7 @@ library
, http-client
, http-media
, http-types
, iproute
, lens
, lucid
, megaparsec
Expand All @@ -281,6 +282,7 @@ library
, natural-transformation
, network
, network-simple
, network-udp
, network-uri
, nonempty-containers
, open-browser
Expand Down Expand Up @@ -462,6 +464,7 @@ test-suite parser-typechecker-tests
, http-client
, http-media
, http-types
, iproute
, lens
, lucid
, megaparsec
Expand All @@ -475,6 +478,7 @@ test-suite parser-typechecker-tests
, natural-transformation
, network
, network-simple
, network-udp
, network-uri
, nonempty-containers
, open-browser
Expand Down
1 change: 1 addition & 0 deletions stack.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ extra-deps:
- lsp-2.2.0.0@sha256:82fbf4b69d94d8d22543be71f89986b3e90050032d671fb3de3f8253ea1e5b6f,3550
- lsp-types-2.0.2.0@sha256:a9a51c3cea0726d91fe63fa0670935ee720f7b31bc3f3b33b2483fc538152677,29421
- row-types-1.0.1.2@sha256:4d4c7cb95d06a32b28ba977852d52a26b4c1f695ef083a6fd874ab6d79933b64,3071
- network-udp-0.0.0

ghc-options:
# All packages
Expand Down
7 changes: 7 additions & 0 deletions stack.yaml.lock
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,13 @@ packages:
size: 1060
original:
hackage: row-types-1.0.1.2@sha256:4d4c7cb95d06a32b28ba977852d52a26b4c1f695ef083a6fd874ab6d79933b64,3071
- completed:
hackage: network-udp-0.0.0@sha256:408d2d4fa1a25e49e95752ee124cca641993404bb133ae10fb81daef22d876ae,1075
pantry-tree:
sha256: ee19a66c9d420861c5cc1dfad3210e2a53cdc6088ff3dd90b44f7961f5caebee
size: 284
original:
hackage: network-udp-0.0.0
snapshots:
- completed:
sha256: 5a59b2a405b3aba3c00188453be172b85893cab8ebc352b1ef58b0eae5d248a2
Expand Down
1 change: 1 addition & 0 deletions unison-cli/package.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ dependencies:
- mtl
- network-uri
- network-simple
- network-udp
- network
- co-log-core
- uri-encode
Expand Down
Loading