Skip to content

Commit

Permalink
Merge pull request #1407 from iotaledger/develop
Browse files Browse the repository at this point in the history
Merge v0.7.0 changes to master
  • Loading branch information
capossele authored Jun 2, 2021
2 parents 5f8e178 + b1478a8 commit a05c532
Show file tree
Hide file tree
Showing 94 changed files with 6,239 additions and 896 deletions.
1 change: 1 addition & 0 deletions .github/workflows/gh-mdbook.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: GoShimmer mdBook

on:
workflow_dispatch:
push:
branches:
- develop
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -287,3 +287,31 @@ jobs:
with:
name: ${{ env.TEST_NAME }}
path: tools/integration-tests/logs

diagnostics:
name: diagnostics
env:
TEST_NAME: diagnostics
runs-on: ubuntu-latest
steps:

- name: Check out code
uses: actions/checkout@v2

- name: Build GoShimmer image
run: docker build -t iotaledger/goshimmer .

- name: Run integration tests
run: docker-compose -f tools/integration-tests/tester/docker-compose.yml up --abort-on-container-exit --exit-code-from tester --build

- name: Create logs from tester
if: always()
run: |
docker logs tester &> tools/integration-tests/logs/tester.log
- name: Save logs as artifacts
if: always()
uses: actions/upload-artifact@v1
with:
name: ${{ env.TEST_NAME }}
path: tools/integration-tests/logs
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,15 @@
# v0.7.0 - 2021-06-02
* Add client diagnostic API
* Improve docs and tutorials
* Improve congestion control
* Improve dashboard
* Improve Grafana dashboard
* Limit chat fields length
* Fix several bugs
* Reset snapshot file
* Update JS dependencies
* **Breaking**: bumps network and database versions

# v0.6.4 - 2021-05-29
* Add simple chat dApp
* Improve client lib
Expand Down
16 changes: 5 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,24 +30,18 @@

## About

This repository is where the IOTA Foundation's Research Department tests the Coordicide modules to study and evaluate their performance.
This repository, called GoShimmer, is where the IOTA Foundation's Research Department tests the IOTA 2.0 modules to study and evaluate their performance.

The aim of this open repository is to give the community the opportunity to follow developments, take part in testing, and learn more about [Coordicide](https://coordicide.iota.org/).
GoShimmer is first and foremost a research prototype. As such, braking changes can often happen. We invite researchers and developers to make use of this project as you see fit. Running experiments, test out new ideas, build PoC are all very welcome initiatives.

For a documentation, including tutorials and resources, we refer to the [Documentation](http://goshimmer.docs.iota.org/) (Github [link](https://github.com/iotaledger/goshimmer/blob/develop/docs/SUMMARY.md)).

**Note:** You can find details about future development plans on our [roadmap](https://roadmap.iota.org).

## Design
The code in GoShimmer is modular, where each module represents either one of the [Coordicide components](https://coordicide.iota.org/) or a basic node function such as the gossip, ledger state, API just to mention a few.

![Layers](images/layers.jpg)

GoShimmer modularity is based on a combination of event-driven and layer-based approaches.
The code in GoShimmer is modular, where each module represents either one of the *IOTA 2.0 components* or a basic node function such as the gossip, ledger state, and API - just to mention a few.

Each module is defined in the `packages` directory, with the exceptions for the dApps (e.g., value transfer, network delay and the faucet) that are under the `dapps` folder. Each module can be enabled using the `plugins` directory.
![Layers](docs/protocol_specification/layers.png)

**Note:** See the `main.go` file to see which plugins are currently supported.
GoShimmer's modularity is based on a combination of event-driven and layer-based approaches.

## Client-Library and HTTP API reference

Expand Down
133 changes: 133 additions & 0 deletions client/diagnostics.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
package client

import (
"encoding/csv"
"fmt"
"net/http"

"github.com/iotaledger/goshimmer/plugins/webapi/tools"
)

// GetDiagnosticsMessages runs full message diagnostics
// Returns CSV with the following fields:
//
// ID IssuerID IssuerPublicKey IssuanceTime ArrivalTime SolidTime ScheduledTime BookedTime OpinionFormedTime
// FinalizedTime StrongParents WeakParents StrongApprovers WeakApprovers BranchID InclusionState Scheduled Booked
// Eligible Invalid Finalized Rank IsPastMarker PastMarkers PMHI PMLI FutureMarkers FMHI FMLI PayloadType TransactionID
// PayloadOpinionFormed TimestampOpinionFormed MessageOpinionFormed MessageOpinionTriggered TimestampOpinion
// TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsMessages() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticMessages)
}

// GetDiagnosticsFirstWeakMessageReferences runs diagnostics over weak references only.
// Returns CSV with the following fields:
//
// ID IssuerID IssuerPublicKey IssuanceTime ArrivalTime SolidTime ScheduledTime BookedTime OpinionFormedTime
// FinalizedTime StrongParents WeakParents StrongApprovers WeakApprovers BranchID InclusionState Scheduled Booked
// Eligible Invalid Finalized Rank IsPastMarker PastMarkers PMHI PMLI FutureMarkers FMHI FMLI PayloadType TransactionID
// PayloadOpinionFormed TimestampOpinionFormed MessageOpinionFormed MessageOpinionTriggered TimestampOpinion
// TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsFirstWeakMessageReferences() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsFirstWeakMessageReferences)
}

// GetDiagnosticsMessagesByRank run diagnostics for messages whose markers are equal or above a certain rank
// Returns CSV with the following fields:
//
// ID IssuerID IssuerPublicKey IssuanceTime ArrivalTime SolidTime ScheduledTime BookedTime OpinionFormedTime
// FinalizedTime StrongParents WeakParents StrongApprovers WeakApprovers BranchID InclusionState Scheduled Booked
// Eligible Invalid Finalized Rank IsPastMarker PastMarkers PMHI PMLI FutureMarkers FMHI FMLI PayloadType TransactionID
// PayloadOpinionFormed TimestampOpinionFormed MessageOpinionFormed MessageOpinionTriggered TimestampOpinion
// TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsMessagesByRank(rank uint64) (*csv.Reader, error) {
return api.diagnose(fmt.Sprintf("%s?rank=%d", tools.RouteDiagnosticMessages, rank))
}

// GetDiagnosticsUtxoDag runs diagnostics over utxo dag.
// Returns csv with the following fields:
//
// ID,IssuanceTime,SolidTime,OpinionFormedTime,AccessManaPledgeID,ConsensusManaPledgeID,Inputs,Outputs,Attachments,
// BranchID,BranchLiked,BranchMonotonicallyLiked,Conflicting,InclusionState,Finalized,LazyBooked,Liked,LoK,FCOB1Time,
// FCOB2Time
func (api *GoShimmerAPI) GetDiagnosticsUtxoDag() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsUtxoDag)
}

// GetDiagnosticsBranches runs diagnostics over branches.
// Returns csv with the following fields:
//
// ID,ConflictSet,IssuanceTime,SolidTime,OpinionFormedTime,Liked,MonotonicallyLiked,InclusionState,Finalized,
// LazyBooked,TransactionLiked
func (api *GoShimmerAPI) GetDiagnosticsBranches() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsBranches)
}

// GetDiagnosticsLazyBookedBranches runs diagnostics over lazy booked branches.
// Returns csv with the following fields:
//
// ID,ConflictSet,IssuanceTime,SolidTime,OpinionFormedTime,Liked,MonotonicallyLiked,InclusionState,Finalized,
// LazyBooked,TransactionLiked
func (api *GoShimmerAPI) GetDiagnosticsLazyBookedBranches() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsLazyBookedBranches)
}

// GetDiagnosticsInvalidBranches runs diagnostics over invalid branches.
// Returns csv with the following fields:
//
// ID,ConflictSet,IssuanceTime,SolidTime,OpinionFormedTime,Liked,MonotonicallyLiked,InclusionState,Finalized,
// LazyBooked,TransactionLiked
func (api *GoShimmerAPI) GetDiagnosticsInvalidBranches() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsInvalidBranches)
}

// GetDiagnosticsTips runs diagnostics over tips
// Returns csv with the following fields:
//
// tipType,ID,IssuerID,IssuerPublicKey,IssuanceTime,ArrivalTime,SolidTime,ScheduledTime,BookedTime,OpinionFormedTime,
// FinalizedTime,StrongParents,WeakParents,StrongApprovers,WeakApprovers,BranchID,InclusionState,Scheduled,Booked,
// Eligible,Invalid,Finalized,Rank,IsPastMarker,PastMarkers,PMHI,PMLI,FutureMarkers,FMHI,FMLI,PayloadType,TransactionID,
// PayloadOpinionFormed,TimestampOpinionFormed,MessageOpinionFormed,MessageOpinionTriggered,TimestampOpinion,TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsTips() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsTips)
}

// GetDiagnosticsStrongTips runs diagnostics over strong tips
// Returns csv with the following fields:
//
// tipType,ID,IssuerID,IssuerPublicKey,IssuanceTime,ArrivalTime,SolidTime,ScheduledTime,BookedTime,OpinionFormedTime,
// FinalizedTime,StrongParents,WeakParents,StrongApprovers,WeakApprovers,BranchID,InclusionState,Scheduled,Booked,
// Eligible,Invalid,Finalized,Rank,IsPastMarker,PastMarkers,PMHI,PMLI,FutureMarkers,FMHI,FMLI,PayloadType,TransactionID,
// PayloadOpinionFormed,TimestampOpinionFormed,MessageOpinionFormed,MessageOpinionTriggered,TimestampOpinion,TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsStrongTips() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsStrongTips)
}

// GetDiagnosticsWeakTips runs diagnostics over weak tips
// Returns csv with the following fields:
//
// tipType,ID,IssuerID,IssuerPublicKey,IssuanceTime,ArrivalTime,SolidTime,ScheduledTime,BookedTime,OpinionFormedTime,
// FinalizedTime,StrongParents,WeakParents,StrongApprovers,WeakApprovers,BranchID,InclusionState,Scheduled,Booked,
// Eligible,Invalid,Finalized,Rank,IsPastMarker,PastMarkers,PMHI,PMLI,FutureMarkers,FMHI,FMLI,PayloadType,TransactionID,
// PayloadOpinionFormed,TimestampOpinionFormed,MessageOpinionFormed,MessageOpinionTriggered,TimestampOpinion,TimestampLoK
func (api *GoShimmerAPI) GetDiagnosticsWeakTips() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsWeakTips)
}

// GetDiagnosticsDRNG runs diagnostics for DRNG
// Returns csv with the following fields:
//
// ID,IssuerID,IssuerPublicKey,IssuanceTime,ArrivalTime,SolidTime,ScheduledTime,BookedTime,OpinionFormedTime,
// dRNGPayloadType,InstanceID,Round,PreviousSignature,Signature,DistributedPK
func (api *GoShimmerAPI) GetDiagnosticsDRNG() (*csv.Reader, error) {
return api.diagnose(tools.RouteDiagnosticsDRNG)
}

// run an api call on a certain route and return a csv
func (api *GoShimmerAPI) diagnose(route string) (*csv.Reader, error) {
reader := &csv.Reader{}
if err := api.do(http.MethodGet, route, nil, reader); err != nil {
return nil, err
}
return reader, nil
}
16 changes: 14 additions & 2 deletions client/lib.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,15 @@
package client

import (
"bufio"
"bytes"
"context"
"encoding/csv"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"

"github.com/cockroachdb/errors"
)
Expand All @@ -28,7 +31,9 @@ var (
)

const (
contentType = "Content-Type"
contentTypeJSON = "application/json"
contentTypeCSV = "text/csv"
)

// Option is a function which sets the given option.
Expand Down Expand Up @@ -99,9 +104,16 @@ func interpretBody(res *http.Response, decodeTo interface{}) error {
defer res.Body.Close()

if res.StatusCode == http.StatusOK || res.StatusCode == http.StatusCreated {
return json.Unmarshal(resBody, decodeTo)
switch contType := res.Header.Get(contentType); {
case strings.HasPrefix(contType, contentTypeJSON):
return json.Unmarshal(resBody, decodeTo)
case strings.HasPrefix(contType, contentTypeCSV):
*decodeTo.(*csv.Reader) = *csv.NewReader(bufio.NewReader(bytes.NewReader(resBody)))
return nil
default:
return fmt.Errorf("Can't decode %s content-type", contType)
}
}

errRes := &errorresponse{}
if err := json.Unmarshal(resBody, errRes); err != nil {
return fmt.Errorf("unable to read error from response body: %w repsonseBody: %s", err, resBody)
Expand Down
2 changes: 1 addition & 1 deletion client/wallet/wallet.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const (
DefaultConfirmationTimeout = 150000 // in ms
milliSeconds = 1000 // miliseconds in a second
// DefaultAssetRegistryNetwork is the default asset registry network.
DefaultAssetRegistryNetwork = "test"
DefaultAssetRegistryNetwork = "nectar"
)

// ErrTooManyOutputs is an error returned when the number of outputs/inputs exceeds the protocol wide constant
Expand Down
25 changes: 21 additions & 4 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,11 @@ http://goshimmer.docs.iota.org/
- [Obtain tokens](./tutorials/request_funds.md)
- [Wallet library](./tutorials/wallet.md)
- [Write a dApp](./tutorials/dApp.md)
- [Manual peering](./tutorials/manual_peering.md)
- [Create a static identity](./tutorials/static_identity.md)
- [Set up a custom dRNG committee](./tutorials/custom_dRNG.md)
- [Set up the Monitoring Dashboard](./tutorials/monitoring.md)
- [How to create and send transactions](./tutorials/send_transaction.md)

- [Implementation design](./implementation_design.md)
- [Event driven model](./implementation_design/event_driven_model.md)
Expand All @@ -26,18 +28,33 @@ http://goshimmer.docs.iota.org/
- [Object storage](./implementation_design/object_storage.md)

- [Protocol specification](./protocol_specification.md)
- [Protocol high level overview](./protocol_speficiation/protocol.md)
- [Protocol high level overview](./protocol_specification/protocol.md)
- [Components](./protocol_specification/components.md)
- [Tangle](./protocol_specification/tangle.md)
- [Autopeering](./protocol_specification/autopeering.md)
- [Mana](./protocol_specification/mana.md)
- [Congestion Control](./protocol_specification/congestion_control.md)
- [Consensus mechanism](./protocol_specification/consensus_mechanism.md)
- [UTXO and Ledgerstate](./protocol_specification/ledgerstate.md)
- [Advanced Outputs (Experimental)](./protocol_specification/advanced_outputs.md)
- [Markers](./protocol_specification/markers.md)
- [Layers](./protocol_specification/layers.md)
- [Glossary](./protocol_specification/glossary.md)

- [API](./api.md)
- [Client Lib](./apis/api.md)
- [WebAPI](./apis/webAPI.md)
- [Mana](./apis/mana.md)
- [Node info](./apis/info.md)
- [Autopeering](./apis/autopeering.md)
- [Manual peering](./apis/manual_peering.md)
- [Communication layer](./apis/communication.md)
- [Ledgerstate](./apis/ledgerstate.md)
- [Mana](./apis/mana.md)
- [dRNG](./apis/dRNG.md)
- [Communication](./apis/communication.md)
- [Snapshot](./apis/snapshot.md)
- [Faucet](./apis/faucet.md)
- [Spammer](./apis/spammer.md)
- [Tools](./apis/tools.md)

- [Tooling](./tooling.md)
- [Docker private network](./tooling/docker_private_network.md)
Expand All @@ -47,5 +64,5 @@ http://goshimmer.docs.iota.org/
- [How to do a release](./teamresources/release.md)
- [Code Guidelines](./teamresources/guidelines.md)
- [Local development](./teamresources/local_development.md)
- [Modify the Analysis dashboard](../plugins/analysis/dashboard/frontend/README.md)
- [Modify the Analysis dashboard](./teamresources/analysis_dashboard.md)

2 changes: 1 addition & 1 deletion docs/apis/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ goshimAPI := client.NewGoShimmerAPI("http://mynode:8080")

Optionally, define your own `http.Client` to use, in order for example to define custom timeouts:
```
goshimAPI := client.NewGoShimmerAPI("http://mynode:8080", http.Client{Timeout: 30 * time.Second})
goshimAPI := client.NewGoShimmerAPI("http://mynode:8080", client.WithHTTPClient{Timeout: 30 * time.Second})
```

#### A note about errors
Expand Down
Loading

0 comments on commit a05c532

Please sign in to comment.