diff --git a/404.html b/404.html old mode 100644 new mode 100755 index 88e06b0..19b8350 --- a/404.html +++ b/404.html @@ -1 +1 @@ - CrossDB

404 - Not found

\ No newline at end of file + CrossDB

404 - Not found

\ No newline at end of file diff --git a/CHANGELOG/index.html b/CHANGELOG/index.html new file mode 100755 index 0000000..ca83ed3 --- /dev/null +++ b/CHANGELOG/index.html @@ -0,0 +1 @@ + Change Log - CrossDB
Skip to content

Change Log

0.6.0 (2024-08-18)

  • Initial refactor release

  • This project was redesigned and rewritten from scratch for over one year

  • Standard RDBMS model
  • New SQL APIs which can support more language
  • MySQL style SQL and shell, which will be easy to study.

0.5.0 (2023-06-26)

Features

  • CrossDB command line tool crossdb-cli is released
  • Optimize insert/update/query/delete performance
  • Add new API cross_matchCreate and cross_matchFree
  • DML APIs supports cross_fields_h and cross_match_h

Bug Fixes

0.4.0 (2023-06-20)

Features

  • Support FreeBSD(X64)
  • Optimize insert/update/query/delete performance
  • Add new API cross_fieldsCreate and cross_fieldsFree

Bug Fixes

0.3.0 (2023-06-13)

Features

  • Support MacOS (X64 and ARM64)
  • Change CROSS_DB_XXX to CROSS_XXX

Bug Fixes

  • cross_dbTblCreate flags CROSS_DB_RBTREE doesn't create Primary Key Index type correctly

0.2.0 (2023-06-07)

Features

  • Support Windows
  • Support Linux ARM64

Bug Fixes

0.1.0 (2023-06-03)

  • Initial release

Features

Bug Fixes

\ No newline at end of file diff --git a/CNAME b/CNAME old mode 100644 new mode 100755 diff --git a/about/about/index.html b/about/about/index.html deleted file mode 100644 index 1eea3aa..0000000 --- a/about/about/index.html +++ /dev/null @@ -1 +0,0 @@ - About US - CrossDB
Skip to content

About CrossDB

CrossDB was developed from 2019. It's designed to be a high-performance embedded database for application, OLTP or embedded system.

To improve development efficiency, more and more features are designed and developed, and it's really an amazing tools for developers.

Our mission is to deliver fantastic tools for embedded database application, high-frequency OLTP and embedded system development with a high-performance embedded database, database-driven development framework and distribution development solution.

CrossDB is very easy to use, high efficiency, powerful and fast enough to help you refactor your project, reduce proudcut time to market and make your project successful.

Contact Us

support@crossdb.org

\ No newline at end of file diff --git a/about/buy/index.html b/about/buy/index.html old mode 100644 new mode 100755 index dc078df..45b2eb0 --- a/about/buy/index.html +++ b/about/buy/index.html @@ -1,6 +1,7 @@ - How to Buy - CrossDB
Skip to content

How to Buy

\ No newline at end of file + + +Buy Bronze Version now! +Buy Silver Version now! +Buy Gold Version now! +Buy Gold Version now! +Buy Platinum Version now! +Buy Diamond Version now! + + + +:material-arrow-right-circle:{ .xdb-arrow } We also have Volume Discounts when you buy. If you need more licenses than shown here, please contact us for a quote. +-->
\ No newline at end of file diff --git a/about/community/index.html b/about/community/index.html deleted file mode 100644 index 76c3a83..0000000 --- a/about/community/index.html +++ /dev/null @@ -1,31 +0,0 @@ - Community - CrossDB
Skip to content

Community

Report Bugs

Github Issues

Require Fetures

Github Fetures

Discussion

Github Discussion

Comments

\ No newline at end of file diff --git a/about/faq/index.html b/about/faq/index.html deleted file mode 100644 index 15b4d0d..0000000 --- a/about/faq/index.html +++ /dev/null @@ -1,31 +0,0 @@ - FAQ - CrossDB
Skip to content

FAQ

Why CrossDB

Comments

\ No newline at end of file diff --git a/about/privacy/index.html b/about/privacy/index.html old mode 100644 new mode 100755 index 9005ae7..734481b --- a/about/privacy/index.html +++ b/about/privacy/index.html @@ -1 +1 @@ - Privacy Notice - CrossDB
Skip to content

Privacy Notice

Introduction

CrossDB Software has designed its website so that, in general, you can browse the CrossDB website without being asked for any personal information. Exceptions to this are when you specifically ask for information, request an evaluation copy of our software or download CrossDB trial software.

This policy covers how CrossDB Software treats personal information that CrossDB may collect and receive via the CrossDB website. Your use of the CrossDB website constitutes acceptance of and agreement with the CrossDB Software Privacy Policy. CrossDB reserves the right to change the CrossDB Software Privacy Policy at any time without notice.

Information collection and use

CrossDB collects personal information when you request for information, request an evaluation of our software or when you download trial software. This information includes information about you such as name, business name and address, telephone and fax numbers, email address, operating system, and hardware environment. Providing information is voluntary. You may provide this information by filling out an on-line form, by email or other communication to CrossDB. However, you may not receive the product information, software evaluation or downloads if you choose not to provide the requested information.

CrossDB Software's use of the information you provided will be to send you the information that you requested. CrossDB Software may also use the information to provide you with additional information about our products and services or market research. The information may be used to follow up with you regarding information supplied, software you downloaded or evaluated.

CrossDB does not sell information collected from users of CrossDB website to other organizations. CrossDB may share the information collected with its subsidiaries and its affiliates such as partners and distributors.

General Information

CrossDB web servers record standard information about CrossDB Web accesses such as IP addresses, your server name, browser type, etc. when you visit our website. CrossDB uses this for information about usage at CrossDB Software's website. The information submitted to CrossDB Software via the Web may be processed, stored and used outside the country where it was submitted.

Links to third party websites are provided for your convenience. By using these links, you will no longer be on the CrossDB Software website and the CrossDB Software Privacy Policy will no longer apply.

Contact us

If you have any question about the information you provided to us or about the CrossDB Software Privacy Policy, please contact us at wang_junchuan@163.com

\ No newline at end of file + Privacy Notice - CrossDB
Skip to content

Privacy Notice

Introduction

CrossDB Software has designed its website so that, in general, you can browse the CrossDB website without being asked for any personal information. Exceptions to this are when you specifically ask for information, request an evaluation copy of our software or download CrossDB trial software.

This policy covers how CrossDB Software treats personal information that CrossDB may collect and receive via the CrossDB website. Your use of the CrossDB website constitutes acceptance of and agreement with the CrossDB Software Privacy Policy. CrossDB reserves the right to change the CrossDB Software Privacy Policy at any time without notice.

Information collection and use

CrossDB collects personal information when you request for information, request an evaluation of our software or when you download trial software. This information includes information about you such as name, business name and address, telephone and fax numbers, email address, operating system, and hardware environment. Providing information is voluntary. You may provide this information by filling out an on-line form, by email or other communication to CrossDB. However, you may not receive the product information, software evaluation or downloads if you choose not to provide the requested information.

CrossDB Software's use of the information you provided will be to send you the information that you requested. CrossDB Software may also use the information to provide you with additional information about our products and services or market research. The information may be used to follow up with you regarding information supplied, software you downloaded or evaluated.

CrossDB does not sell information collected from users of CrossDB website to other organizations. CrossDB may share the information collected with its subsidiaries and its affiliates such as partners and distributors.

General Information

CrossDB web servers record standard information about CrossDB Web accesses such as IP addresses, your server name, browser type, etc. when you visit our website. CrossDB uses this for information about usage at CrossDB Software's website. The information submitted to CrossDB Software via the Web may be processed, stored and used outside the country where it was submitted.

Links to third party websites are provided for your convenience. By using these links, you will no longer be on the CrossDB Software website and the CrossDB Software Privacy Policy will no longer apply.

Contact us

If you have any question about the information you provided to us or about the CrossDB Software Privacy Policy, please contact us at support@crossdb.org

\ No newline at end of file diff --git a/about/refund/index.html b/about/refund/index.html old mode 100644 new mode 100755 index 8ad7660..d5147a6 --- a/about/refund/index.html +++ b/about/refund/index.html @@ -1 +1 @@ - Refund Policy - CrossDB
Skip to content

Refund Policy

Thank you for using CrossDB Embedded Database! We offer refund and/or exchange within the first 30 days of your purchase, if 30 days have passed since your purchase, you will not be offered a refund and/or exchange of any kind.

\ No newline at end of file + Refund Policy - CrossDB
Skip to content

Refund Policy

Thank you for using CrossDB Database! We don't offer refund, please try it first then subscribe the product by month or year. You can cancel the subscription anytime as you want.

\ No newline at end of file diff --git a/about/terms/index.html b/about/terms/index.html old mode 100644 new mode 100755 index d14230a..a7d5252 --- a/about/terms/index.html +++ b/about/terms/index.html @@ -1 +1 @@ - Terms of Service - CrossDB
Skip to content

Terms of Service

CrossDB provides this Website for your information and use, subject to the terms set forth below. By accessing this Website, you agree to accept the following terms.

General terms

In no event will CrossDB be liable for any damages including, without limitation, indirect, incidental, consequential, special or exemplary damages, that arise out of or relate to the use of or inability to use the CrossDB Website, even if CrossDB has been advised of the possibility of such damages.

Any and all information provided on CrossDB’s Website is provided “as is,” with no warranty as to accuracy or content.

CrossDB does not adopt or endorse the views of any third party Websites linked to or from the CrossDB Website.

CrossDB reserves the right to change, modify, add or remove any of these terms at any time.

\ No newline at end of file + Terms of Service - CrossDB
Skip to content

Terms of Service

CrossDB provides this Website for your information and use, subject to the terms set forth below. By accessing this Website, you agree to accept the following terms.

General terms

In no event will CrossDB be liable for any damages including, without limitation, indirect, incidental, consequential, special or exemplary damages, that arise out of or relate to the use of or inability to use the CrossDB Website, even if CrossDB has been advised of the possibility of such damages.

Any and all information provided on CrossDB’s Website is provided “as is,” with no warranty as to accuracy or content.

CrossDB does not adopt or endorse the views of any third party Websites linked to or from the CrossDB Website.

CrossDB reserves the right to change, modify, add or remove any of these terms at any time.

\ No newline at end of file diff --git a/admin/data-backup/index.html b/admin/data-backup/index.html new file mode 100755 index 0000000..75491e2 --- /dev/null +++ b/admin/data-backup/index.html @@ -0,0 +1,41 @@ + Data Backup - CrossDB
Skip to content

Data Backup

Dump with DUMP statement

  • Dump everything
DUMP DATABASE school INTO 'school.sql';
+
xdb_exec (pConn, "DUMP DATABASE school INTO 'school.sql'");
+
  • Dump schema only
DUMP DATABASE school NODATA INTO 'school.sql';
+
xdb_exec (pConn, "DUMP DATABASE school NODATA INTO 'school.sql'");
+
  • Dump data only
DUMP DATABASE school NODROP NOCREATE INTO 'school.sql';
+
xdb_exec (pConn, "DUMP DATABASE school NODROP NOCREATE INTO 'school.sql'");
+

Dump with crossdb tool

  • Dump everything
crossdb -e "DUMP DATABASE" school > school.sql
+
crossdb -e "DUMP DATABASE school INTO 'school.sql'" school
+
  • Dump schema only
crossdb -e "DUMP DATABASE school NODATA INTO 'school.sql'" school
+
  • Dump data only
crossdb -e "DUMP DATABASE school NODROP NOCREATE INTO 'school.sql'" school
+

Comments

\ No newline at end of file diff --git a/admin/data-restore/index.html b/admin/data-restore/index.html new file mode 100755 index 0000000..8411c9d --- /dev/null +++ b/admin/data-restore/index.html @@ -0,0 +1,36 @@ + Data Restore - CrossDB
Skip to content

Data Restore

Load with SOURCE statement

SOURCE 'school.sql'
+
xdb_exec (pConn, "SOURCE 'school.sql'");
+

Load with CrossDB tool

crossdb school < school.sql
+
cat school.sql | crossdb school
+
crossdb -e "SOURCE 'school.sql'" school
+

Comments

\ No newline at end of file diff --git a/admin/shell/index.html b/admin/shell/index.html new file mode 100755 index 0000000..be47688 --- /dev/null +++ b/admin/shell/index.html @@ -0,0 +1,92 @@ + CrossDB Shell - CrossDB
Skip to content

CrossDB Shell

CrossDB tool

This tool can

  • Open and operate local database
  • Connect and operate remote database
  • Work as standalone database server
$ crossdb -h
+crossdb: option requires an argument -- 'h'
+Usage: xdb-cli [OPTIONS] [[path/]db_name]
+  -h                        Show this help
+  -S                        Server: Start in server mode
+  -h <ip>                   IP address to bind to or connect to
+  -P <port>                 Port to listen or connect
+  -D <datadir>              Server: Data directory to store databases
+  -u <user>                 Client user
+  -p                        Client password
+  -e <sql>                  Execute SQL statements and quite
+

Open DB

crossdb school
+

Open DB and execute command

crossdb -e 'SELECT * FROM student WHERE id=1; SELECT * FROM student WHERE age=10' school
+

Embedded shell

You program can enter interactive shell use the SHELL statement.

SHELL
+
xdb_exec (pConn, "SHELL");
+

Auto completion

Use TAB to do auto completion for SQL statements, keywords, db name, table name, field name, etc.

XDB> <TAB>
+CREATE      Create database, table, index, etc
+DROP        Drop database, table, index, etc
+ALTER       Alter database, table, index, etc
+SELECT      Select rows from table
+INSERT      Insert rows into table
+UPDATE      Update rows in table
+DELETE      Delete rows from table
+BEGIN       Begin transaction
+COMMIT      Commit transaction
+ROLLBACK    Rollback transaction
+USE         Switch Databae
+SHOW        Show database, table, index, etc
+DESCRIBE    Show Table Schema
+EXPLAIN     Show SELECT statement index selection
+SET         Config parameters
+OPEN        Open database
+CLOSE       Close database
+DUMP        Dump database
+SOURCE      Load SQL file
+SHELL       Enter interactive shell
+HELP        Help
+
XDB> S <TAB>
+SELECT    Select rows from table
+SHOW      Show database, table, index, etc
+SET       Config parameters
+SOURCE    Load SQL file
+SHELL     Enter interactive shell
+XDB> SE <TAB>
+SELECT    Select rows from table
+SET       Config parameters
+XDB> SEL <TAB>
+XDB> SELECT
+
XDB> SELECT <TAB>
+XDB> SELECT * <TAB>
+XDB> SELECT * FROM <TAB>
+student    teacher    book
+XDB> SELECT * FROM s<TAB>
+XDB> SELECT * FROM student
+
XDB> SELECT * FROM student <TAB>
+WHERE       ORDER BY    LIMIT       OFFSET      AND         id          name        age         class       score
+info
+XDB> SELECT * FROM student WHERE i <TAB>
+id      info
+
XDB> SHOW  <TAB>
+DATABASES    TABLES       INDEXES      COLUMNS      CREATE
+

Comments

\ No newline at end of file diff --git a/assets/favicon.png b/assets/favicon.png old mode 100644 new mode 100755 diff --git a/assets/images/favicon.png b/assets/images/favicon.png old mode 100644 new mode 100755 diff --git a/assets/javascripts/baidu-tongji.js b/assets/javascripts/baidu-tongji.js old mode 100644 new mode 100755 diff --git a/assets/javascripts/bundle.fac441b0.min.js b/assets/javascripts/bundle.fac441b0.min.js deleted file mode 100644 index 4bb4cd6..0000000 --- a/assets/javascripts/bundle.fac441b0.min.js +++ /dev/null @@ -1,29 +0,0 @@ -"use strict";(()=>{var Ci=Object.create;var gr=Object.defineProperty;var Ri=Object.getOwnPropertyDescriptor;var ki=Object.getOwnPropertyNames,Ht=Object.getOwnPropertySymbols,Hi=Object.getPrototypeOf,yr=Object.prototype.hasOwnProperty,nn=Object.prototype.propertyIsEnumerable;var rn=(e,t,r)=>t in e?gr(e,t,{enumerable:!0,configurable:!0,writable:!0,value:r}):e[t]=r,P=(e,t)=>{for(var r in t||(t={}))yr.call(t,r)&&rn(e,r,t[r]);if(Ht)for(var r of Ht(t))nn.call(t,r)&&rn(e,r,t[r]);return e};var on=(e,t)=>{var r={};for(var n in e)yr.call(e,n)&&t.indexOf(n)<0&&(r[n]=e[n]);if(e!=null&&Ht)for(var n of Ht(e))t.indexOf(n)<0&&nn.call(e,n)&&(r[n]=e[n]);return r};var Pt=(e,t)=>()=>(t||e((t={exports:{}}).exports,t),t.exports);var Pi=(e,t,r,n)=>{if(t&&typeof t=="object"||typeof t=="function")for(let o of ki(t))!yr.call(e,o)&&o!==r&&gr(e,o,{get:()=>t[o],enumerable:!(n=Ri(t,o))||n.enumerable});return e};var yt=(e,t,r)=>(r=e!=null?Ci(Hi(e)):{},Pi(t||!e||!e.__esModule?gr(r,"default",{value:e,enumerable:!0}):r,e));var sn=Pt((xr,an)=>{(function(e,t){typeof xr=="object"&&typeof an!="undefined"?t():typeof define=="function"&&define.amd?define(t):t()})(xr,function(){"use strict";function e(r){var n=!0,o=!1,i=null,s={text:!0,search:!0,url:!0,tel:!0,email:!0,password:!0,number:!0,date:!0,month:!0,week:!0,time:!0,datetime:!0,"datetime-local":!0};function a(O){return!!(O&&O!==document&&O.nodeName!=="HTML"&&O.nodeName!=="BODY"&&"classList"in O&&"contains"in O.classList)}function f(O){var Qe=O.type,De=O.tagName;return!!(De==="INPUT"&&s[Qe]&&!O.readOnly||De==="TEXTAREA"&&!O.readOnly||O.isContentEditable)}function c(O){O.classList.contains("focus-visible")||(O.classList.add("focus-visible"),O.setAttribute("data-focus-visible-added",""))}function u(O){O.hasAttribute("data-focus-visible-added")&&(O.classList.remove("focus-visible"),O.removeAttribute("data-focus-visible-added"))}function p(O){O.metaKey||O.altKey||O.ctrlKey||(a(r.activeElement)&&c(r.activeElement),n=!0)}function m(O){n=!1}function d(O){a(O.target)&&(n||f(O.target))&&c(O.target)}function h(O){a(O.target)&&(O.target.classList.contains("focus-visible")||O.target.hasAttribute("data-focus-visible-added"))&&(o=!0,window.clearTimeout(i),i=window.setTimeout(function(){o=!1},100),u(O.target))}function v(O){document.visibilityState==="hidden"&&(o&&(n=!0),Q())}function Q(){document.addEventListener("mousemove",N),document.addEventListener("mousedown",N),document.addEventListener("mouseup",N),document.addEventListener("pointermove",N),document.addEventListener("pointerdown",N),document.addEventListener("pointerup",N),document.addEventListener("touchmove",N),document.addEventListener("touchstart",N),document.addEventListener("touchend",N)}function B(){document.removeEventListener("mousemove",N),document.removeEventListener("mousedown",N),document.removeEventListener("mouseup",N),document.removeEventListener("pointermove",N),document.removeEventListener("pointerdown",N),document.removeEventListener("pointerup",N),document.removeEventListener("touchmove",N),document.removeEventListener("touchstart",N),document.removeEventListener("touchend",N)}function N(O){O.target.nodeName&&O.target.nodeName.toLowerCase()==="html"||(n=!1,B())}document.addEventListener("keydown",p,!0),document.addEventListener("mousedown",m,!0),document.addEventListener("pointerdown",m,!0),document.addEventListener("touchstart",m,!0),document.addEventListener("visibilitychange",v,!0),Q(),r.addEventListener("focus",d,!0),r.addEventListener("blur",h,!0),r.nodeType===Node.DOCUMENT_FRAGMENT_NODE&&r.host?r.host.setAttribute("data-js-focus-visible",""):r.nodeType===Node.DOCUMENT_NODE&&(document.documentElement.classList.add("js-focus-visible"),document.documentElement.setAttribute("data-js-focus-visible",""))}if(typeof window!="undefined"&&typeof document!="undefined"){window.applyFocusVisiblePolyfill=e;var t;try{t=new CustomEvent("focus-visible-polyfill-ready")}catch(r){t=document.createEvent("CustomEvent"),t.initCustomEvent("focus-visible-polyfill-ready",!1,!1,{})}window.dispatchEvent(t)}typeof document!="undefined"&&e(document)})});var cn=Pt(Er=>{(function(e){var t=function(){try{return!!Symbol.iterator}catch(c){return!1}},r=t(),n=function(c){var u={next:function(){var p=c.shift();return{done:p===void 0,value:p}}};return r&&(u[Symbol.iterator]=function(){return u}),u},o=function(c){return encodeURIComponent(c).replace(/%20/g,"+")},i=function(c){return decodeURIComponent(String(c).replace(/\+/g," "))},s=function(){var c=function(p){Object.defineProperty(this,"_entries",{writable:!0,value:{}});var m=typeof p;if(m!=="undefined")if(m==="string")p!==""&&this._fromString(p);else if(p instanceof c){var d=this;p.forEach(function(B,N){d.append(N,B)})}else if(p!==null&&m==="object")if(Object.prototype.toString.call(p)==="[object Array]")for(var h=0;hd[0]?1:0}),c._entries&&(c._entries={});for(var p=0;p1?i(d[1]):"")}})})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Er);(function(e){var t=function(){try{var o=new e.URL("b","http://a");return o.pathname="c d",o.href==="http://a/c%20d"&&o.searchParams}catch(i){return!1}},r=function(){var o=e.URL,i=function(f,c){typeof f!="string"&&(f=String(f)),c&&typeof c!="string"&&(c=String(c));var u=document,p;if(c&&(e.location===void 0||c!==e.location.href)){c=c.toLowerCase(),u=document.implementation.createHTMLDocument(""),p=u.createElement("base"),p.href=c,u.head.appendChild(p);try{if(p.href.indexOf(c)!==0)throw new Error(p.href)}catch(O){throw new Error("URL unable to set base "+c+" due to "+O)}}var m=u.createElement("a");m.href=f,p&&(u.body.appendChild(m),m.href=m.href);var d=u.createElement("input");if(d.type="url",d.value=f,m.protocol===":"||!/:/.test(m.href)||!d.checkValidity()&&!c)throw new TypeError("Invalid URL");Object.defineProperty(this,"_anchorElement",{value:m});var h=new e.URLSearchParams(this.search),v=!0,Q=!0,B=this;["append","delete","set"].forEach(function(O){var Qe=h[O];h[O]=function(){Qe.apply(h,arguments),v&&(Q=!1,B.search=h.toString(),Q=!0)}}),Object.defineProperty(this,"searchParams",{value:h,enumerable:!0});var N=void 0;Object.defineProperty(this,"_updateSearchParams",{enumerable:!1,configurable:!1,writable:!1,value:function(){this.search!==N&&(N=this.search,Q&&(v=!1,this.searchParams._fromString(this.search),v=!0))}})},s=i.prototype,a=function(f){Object.defineProperty(s,f,{get:function(){return this._anchorElement[f]},set:function(c){this._anchorElement[f]=c},enumerable:!0})};["hash","host","hostname","port","protocol"].forEach(function(f){a(f)}),Object.defineProperty(s,"search",{get:function(){return this._anchorElement.search},set:function(f){this._anchorElement.search=f,this._updateSearchParams()},enumerable:!0}),Object.defineProperties(s,{toString:{get:function(){var f=this;return function(){return f.href}}},href:{get:function(){return this._anchorElement.href.replace(/\?$/,"")},set:function(f){this._anchorElement.href=f,this._updateSearchParams()},enumerable:!0},pathname:{get:function(){return this._anchorElement.pathname.replace(/(^\/?)/,"/")},set:function(f){this._anchorElement.pathname=f},enumerable:!0},origin:{get:function(){var f={"http:":80,"https:":443,"ftp:":21}[this._anchorElement.protocol],c=this._anchorElement.port!=f&&this._anchorElement.port!=="";return this._anchorElement.protocol+"//"+this._anchorElement.hostname+(c?":"+this._anchorElement.port:"")},enumerable:!0},password:{get:function(){return""},set:function(f){},enumerable:!0},username:{get:function(){return""},set:function(f){},enumerable:!0}}),i.createObjectURL=function(f){return o.createObjectURL.apply(o,arguments)},i.revokeObjectURL=function(f){return o.revokeObjectURL.apply(o,arguments)},e.URL=i};if(t()||r(),e.location!==void 0&&!("origin"in e.location)){var n=function(){return e.location.protocol+"//"+e.location.hostname+(e.location.port?":"+e.location.port:"")};try{Object.defineProperty(e.location,"origin",{get:n,enumerable:!0})}catch(o){setInterval(function(){e.location.origin=n()},100)}}})(typeof global!="undefined"?global:typeof window!="undefined"?window:typeof self!="undefined"?self:Er)});var qr=Pt((Mt,Nr)=>{/*! - * clipboard.js v2.0.11 - * https://clipboardjs.com/ - * - * Licensed MIT © Zeno Rocha - */(function(t,r){typeof Mt=="object"&&typeof Nr=="object"?Nr.exports=r():typeof define=="function"&&define.amd?define([],r):typeof Mt=="object"?Mt.ClipboardJS=r():t.ClipboardJS=r()})(Mt,function(){return function(){var e={686:function(n,o,i){"use strict";i.d(o,{default:function(){return Ai}});var s=i(279),a=i.n(s),f=i(370),c=i.n(f),u=i(817),p=i.n(u);function m(j){try{return document.execCommand(j)}catch(T){return!1}}var d=function(T){var E=p()(T);return m("cut"),E},h=d;function v(j){var T=document.documentElement.getAttribute("dir")==="rtl",E=document.createElement("textarea");E.style.fontSize="12pt",E.style.border="0",E.style.padding="0",E.style.margin="0",E.style.position="absolute",E.style[T?"right":"left"]="-9999px";var H=window.pageYOffset||document.documentElement.scrollTop;return E.style.top="".concat(H,"px"),E.setAttribute("readonly",""),E.value=j,E}var Q=function(T,E){var H=v(T);E.container.appendChild(H);var I=p()(H);return m("copy"),H.remove(),I},B=function(T){var E=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body},H="";return typeof T=="string"?H=Q(T,E):T instanceof HTMLInputElement&&!["text","search","url","tel","password"].includes(T==null?void 0:T.type)?H=Q(T.value,E):(H=p()(T),m("copy")),H},N=B;function O(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?O=function(E){return typeof E}:O=function(E){return E&&typeof Symbol=="function"&&E.constructor===Symbol&&E!==Symbol.prototype?"symbol":typeof E},O(j)}var Qe=function(){var T=arguments.length>0&&arguments[0]!==void 0?arguments[0]:{},E=T.action,H=E===void 0?"copy":E,I=T.container,q=T.target,Me=T.text;if(H!=="copy"&&H!=="cut")throw new Error('Invalid "action" value, use either "copy" or "cut"');if(q!==void 0)if(q&&O(q)==="object"&&q.nodeType===1){if(H==="copy"&&q.hasAttribute("disabled"))throw new Error('Invalid "target" attribute. Please use "readonly" instead of "disabled" attribute');if(H==="cut"&&(q.hasAttribute("readonly")||q.hasAttribute("disabled")))throw new Error(`Invalid "target" attribute. You can't cut text from elements with "readonly" or "disabled" attributes`)}else throw new Error('Invalid "target" value, use a valid Element');if(Me)return N(Me,{container:I});if(q)return H==="cut"?h(q):N(q,{container:I})},De=Qe;function $e(j){return typeof Symbol=="function"&&typeof Symbol.iterator=="symbol"?$e=function(E){return typeof E}:$e=function(E){return E&&typeof Symbol=="function"&&E.constructor===Symbol&&E!==Symbol.prototype?"symbol":typeof E},$e(j)}function Ei(j,T){if(!(j instanceof T))throw new TypeError("Cannot call a class as a function")}function tn(j,T){for(var E=0;E0&&arguments[0]!==void 0?arguments[0]:{};this.action=typeof I.action=="function"?I.action:this.defaultAction,this.target=typeof I.target=="function"?I.target:this.defaultTarget,this.text=typeof I.text=="function"?I.text:this.defaultText,this.container=$e(I.container)==="object"?I.container:document.body}},{key:"listenClick",value:function(I){var q=this;this.listener=c()(I,"click",function(Me){return q.onClick(Me)})}},{key:"onClick",value:function(I){var q=I.delegateTarget||I.currentTarget,Me=this.action(q)||"copy",kt=De({action:Me,container:this.container,target:this.target(q),text:this.text(q)});this.emit(kt?"success":"error",{action:Me,text:kt,trigger:q,clearSelection:function(){q&&q.focus(),window.getSelection().removeAllRanges()}})}},{key:"defaultAction",value:function(I){return vr("action",I)}},{key:"defaultTarget",value:function(I){var q=vr("target",I);if(q)return document.querySelector(q)}},{key:"defaultText",value:function(I){return vr("text",I)}},{key:"destroy",value:function(){this.listener.destroy()}}],[{key:"copy",value:function(I){var q=arguments.length>1&&arguments[1]!==void 0?arguments[1]:{container:document.body};return N(I,q)}},{key:"cut",value:function(I){return h(I)}},{key:"isSupported",value:function(){var I=arguments.length>0&&arguments[0]!==void 0?arguments[0]:["copy","cut"],q=typeof I=="string"?[I]:I,Me=!!document.queryCommandSupported;return q.forEach(function(kt){Me=Me&&!!document.queryCommandSupported(kt)}),Me}}]),E}(a()),Ai=Li},828:function(n){var o=9;if(typeof Element!="undefined"&&!Element.prototype.matches){var i=Element.prototype;i.matches=i.matchesSelector||i.mozMatchesSelector||i.msMatchesSelector||i.oMatchesSelector||i.webkitMatchesSelector}function s(a,f){for(;a&&a.nodeType!==o;){if(typeof a.matches=="function"&&a.matches(f))return a;a=a.parentNode}}n.exports=s},438:function(n,o,i){var s=i(828);function a(u,p,m,d,h){var v=c.apply(this,arguments);return u.addEventListener(m,v,h),{destroy:function(){u.removeEventListener(m,v,h)}}}function f(u,p,m,d,h){return typeof u.addEventListener=="function"?a.apply(null,arguments):typeof m=="function"?a.bind(null,document).apply(null,arguments):(typeof u=="string"&&(u=document.querySelectorAll(u)),Array.prototype.map.call(u,function(v){return a(v,p,m,d,h)}))}function c(u,p,m,d){return function(h){h.delegateTarget=s(h.target,p),h.delegateTarget&&d.call(u,h)}}n.exports=f},879:function(n,o){o.node=function(i){return i!==void 0&&i instanceof HTMLElement&&i.nodeType===1},o.nodeList=function(i){var s=Object.prototype.toString.call(i);return i!==void 0&&(s==="[object NodeList]"||s==="[object HTMLCollection]")&&"length"in i&&(i.length===0||o.node(i[0]))},o.string=function(i){return typeof i=="string"||i instanceof String},o.fn=function(i){var s=Object.prototype.toString.call(i);return s==="[object Function]"}},370:function(n,o,i){var s=i(879),a=i(438);function f(m,d,h){if(!m&&!d&&!h)throw new Error("Missing required arguments");if(!s.string(d))throw new TypeError("Second argument must be a String");if(!s.fn(h))throw new TypeError("Third argument must be a Function");if(s.node(m))return c(m,d,h);if(s.nodeList(m))return u(m,d,h);if(s.string(m))return p(m,d,h);throw new TypeError("First argument must be a String, HTMLElement, HTMLCollection, or NodeList")}function c(m,d,h){return m.addEventListener(d,h),{destroy:function(){m.removeEventListener(d,h)}}}function u(m,d,h){return Array.prototype.forEach.call(m,function(v){v.addEventListener(d,h)}),{destroy:function(){Array.prototype.forEach.call(m,function(v){v.removeEventListener(d,h)})}}}function p(m,d,h){return a(document.body,m,d,h)}n.exports=f},817:function(n){function o(i){var s;if(i.nodeName==="SELECT")i.focus(),s=i.value;else if(i.nodeName==="INPUT"||i.nodeName==="TEXTAREA"){var a=i.hasAttribute("readonly");a||i.setAttribute("readonly",""),i.select(),i.setSelectionRange(0,i.value.length),a||i.removeAttribute("readonly"),s=i.value}else{i.hasAttribute("contenteditable")&&i.focus();var f=window.getSelection(),c=document.createRange();c.selectNodeContents(i),f.removeAllRanges(),f.addRange(c),s=f.toString()}return s}n.exports=o},279:function(n){function o(){}o.prototype={on:function(i,s,a){var f=this.e||(this.e={});return(f[i]||(f[i]=[])).push({fn:s,ctx:a}),this},once:function(i,s,a){var f=this;function c(){f.off(i,c),s.apply(a,arguments)}return c._=s,this.on(i,c,a)},emit:function(i){var s=[].slice.call(arguments,1),a=((this.e||(this.e={}))[i]||[]).slice(),f=0,c=a.length;for(f;f{"use strict";/*! - * escape-html - * Copyright(c) 2012-2013 TJ Holowaychuk - * Copyright(c) 2015 Andreas Lubbe - * Copyright(c) 2015 Tiancheng "Timothy" Gu - * MIT Licensed - */var rs=/["'&<>]/;Yo.exports=ns;function ns(e){var t=""+e,r=rs.exec(t);if(!r)return t;var n,o="",i=0,s=0;for(i=r.index;i0&&i[i.length-1])&&(c[0]===6||c[0]===2)){r=0;continue}if(c[0]===3&&(!i||c[1]>i[0]&&c[1]=e.length&&(e=void 0),{value:e&&e[n++],done:!e}}};throw new TypeError(t?"Object is not iterable.":"Symbol.iterator is not defined.")}function W(e,t){var r=typeof Symbol=="function"&&e[Symbol.iterator];if(!r)return e;var n=r.call(e),o,i=[],s;try{for(;(t===void 0||t-- >0)&&!(o=n.next()).done;)i.push(o.value)}catch(a){s={error:a}}finally{try{o&&!o.done&&(r=n.return)&&r.call(n)}finally{if(s)throw s.error}}return i}function D(e,t,r){if(r||arguments.length===2)for(var n=0,o=t.length,i;n1||a(m,d)})})}function a(m,d){try{f(n[m](d))}catch(h){p(i[0][3],h)}}function f(m){m.value instanceof et?Promise.resolve(m.value.v).then(c,u):p(i[0][2],m)}function c(m){a("next",m)}function u(m){a("throw",m)}function p(m,d){m(d),i.shift(),i.length&&a(i[0][0],i[0][1])}}function pn(e){if(!Symbol.asyncIterator)throw new TypeError("Symbol.asyncIterator is not defined.");var t=e[Symbol.asyncIterator],r;return t?t.call(e):(e=typeof Ee=="function"?Ee(e):e[Symbol.iterator](),r={},n("next"),n("throw"),n("return"),r[Symbol.asyncIterator]=function(){return this},r);function n(i){r[i]=e[i]&&function(s){return new Promise(function(a,f){s=e[i](s),o(a,f,s.done,s.value)})}}function o(i,s,a,f){Promise.resolve(f).then(function(c){i({value:c,done:a})},s)}}function C(e){return typeof e=="function"}function at(e){var t=function(n){Error.call(n),n.stack=new Error().stack},r=e(t);return r.prototype=Object.create(Error.prototype),r.prototype.constructor=r,r}var It=at(function(e){return function(r){e(this),this.message=r?r.length+` errors occurred during unsubscription: -`+r.map(function(n,o){return o+1+") "+n.toString()}).join(` - `):"",this.name="UnsubscriptionError",this.errors=r}});function Ve(e,t){if(e){var r=e.indexOf(t);0<=r&&e.splice(r,1)}}var Ie=function(){function e(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}return e.prototype.unsubscribe=function(){var t,r,n,o,i;if(!this.closed){this.closed=!0;var s=this._parentage;if(s)if(this._parentage=null,Array.isArray(s))try{for(var a=Ee(s),f=a.next();!f.done;f=a.next()){var c=f.value;c.remove(this)}}catch(v){t={error:v}}finally{try{f&&!f.done&&(r=a.return)&&r.call(a)}finally{if(t)throw t.error}}else s.remove(this);var u=this.initialTeardown;if(C(u))try{u()}catch(v){i=v instanceof It?v.errors:[v]}var p=this._finalizers;if(p){this._finalizers=null;try{for(var m=Ee(p),d=m.next();!d.done;d=m.next()){var h=d.value;try{ln(h)}catch(v){i=i!=null?i:[],v instanceof It?i=D(D([],W(i)),W(v.errors)):i.push(v)}}}catch(v){n={error:v}}finally{try{d&&!d.done&&(o=m.return)&&o.call(m)}finally{if(n)throw n.error}}}if(i)throw new It(i)}},e.prototype.add=function(t){var r;if(t&&t!==this)if(this.closed)ln(t);else{if(t instanceof e){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(r=this._finalizers)!==null&&r!==void 0?r:[]).push(t)}},e.prototype._hasParent=function(t){var r=this._parentage;return r===t||Array.isArray(r)&&r.includes(t)},e.prototype._addParent=function(t){var r=this._parentage;this._parentage=Array.isArray(r)?(r.push(t),r):r?[r,t]:t},e.prototype._removeParent=function(t){var r=this._parentage;r===t?this._parentage=null:Array.isArray(r)&&Ve(r,t)},e.prototype.remove=function(t){var r=this._finalizers;r&&Ve(r,t),t instanceof e&&t._removeParent(this)},e.EMPTY=function(){var t=new e;return t.closed=!0,t}(),e}();var Sr=Ie.EMPTY;function jt(e){return e instanceof Ie||e&&"closed"in e&&C(e.remove)&&C(e.add)&&C(e.unsubscribe)}function ln(e){C(e)?e():e.unsubscribe()}var Le={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var st={setTimeout:function(e,t){for(var r=[],n=2;n0},enumerable:!1,configurable:!0}),t.prototype._trySubscribe=function(r){return this._throwIfClosed(),e.prototype._trySubscribe.call(this,r)},t.prototype._subscribe=function(r){return this._throwIfClosed(),this._checkFinalizedStatuses(r),this._innerSubscribe(r)},t.prototype._innerSubscribe=function(r){var n=this,o=this,i=o.hasError,s=o.isStopped,a=o.observers;return i||s?Sr:(this.currentObservers=null,a.push(r),new Ie(function(){n.currentObservers=null,Ve(a,r)}))},t.prototype._checkFinalizedStatuses=function(r){var n=this,o=n.hasError,i=n.thrownError,s=n.isStopped;o?r.error(i):s&&r.complete()},t.prototype.asObservable=function(){var r=new F;return r.source=this,r},t.create=function(r,n){return new xn(r,n)},t}(F);var xn=function(e){ie(t,e);function t(r,n){var o=e.call(this)||this;return o.destination=r,o.source=n,o}return t.prototype.next=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.next)===null||o===void 0||o.call(n,r)},t.prototype.error=function(r){var n,o;(o=(n=this.destination)===null||n===void 0?void 0:n.error)===null||o===void 0||o.call(n,r)},t.prototype.complete=function(){var r,n;(n=(r=this.destination)===null||r===void 0?void 0:r.complete)===null||n===void 0||n.call(r)},t.prototype._subscribe=function(r){var n,o;return(o=(n=this.source)===null||n===void 0?void 0:n.subscribe(r))!==null&&o!==void 0?o:Sr},t}(x);var Et={now:function(){return(Et.delegate||Date).now()},delegate:void 0};var wt=function(e){ie(t,e);function t(r,n,o){r===void 0&&(r=1/0),n===void 0&&(n=1/0),o===void 0&&(o=Et);var i=e.call(this)||this;return i._bufferSize=r,i._windowTime=n,i._timestampProvider=o,i._buffer=[],i._infiniteTimeWindow=!0,i._infiniteTimeWindow=n===1/0,i._bufferSize=Math.max(1,r),i._windowTime=Math.max(1,n),i}return t.prototype.next=function(r){var n=this,o=n.isStopped,i=n._buffer,s=n._infiniteTimeWindow,a=n._timestampProvider,f=n._windowTime;o||(i.push(r),!s&&i.push(a.now()+f)),this._trimBuffer(),e.prototype.next.call(this,r)},t.prototype._subscribe=function(r){this._throwIfClosed(),this._trimBuffer();for(var n=this._innerSubscribe(r),o=this,i=o._infiniteTimeWindow,s=o._buffer,a=s.slice(),f=0;f0?e.prototype.requestAsyncId.call(this,r,n,o):(r.actions.push(this),r._scheduled||(r._scheduled=ut.requestAnimationFrame(function(){return r.flush(void 0)})))},t.prototype.recycleAsyncId=function(r,n,o){var i;if(o===void 0&&(o=0),o!=null?o>0:this.delay>0)return e.prototype.recycleAsyncId.call(this,r,n,o);var s=r.actions;n!=null&&((i=s[s.length-1])===null||i===void 0?void 0:i.id)!==n&&(ut.cancelAnimationFrame(n),r._scheduled=void 0)},t}(Wt);var Sn=function(e){ie(t,e);function t(){return e!==null&&e.apply(this,arguments)||this}return t.prototype.flush=function(r){this._active=!0;var n=this._scheduled;this._scheduled=void 0;var o=this.actions,i;r=r||o.shift();do if(i=r.execute(r.state,r.delay))break;while((r=o[0])&&r.id===n&&o.shift());if(this._active=!1,i){for(;(r=o[0])&&r.id===n&&o.shift();)r.unsubscribe();throw i}},t}(Dt);var Oe=new Sn(wn);var _=new F(function(e){return e.complete()});function Vt(e){return e&&C(e.schedule)}function Cr(e){return e[e.length-1]}function Ye(e){return C(Cr(e))?e.pop():void 0}function Te(e){return Vt(Cr(e))?e.pop():void 0}function zt(e,t){return typeof Cr(e)=="number"?e.pop():t}var pt=function(e){return e&&typeof e.length=="number"&&typeof e!="function"};function Nt(e){return C(e==null?void 0:e.then)}function qt(e){return C(e[ft])}function Kt(e){return Symbol.asyncIterator&&C(e==null?void 0:e[Symbol.asyncIterator])}function Qt(e){return new TypeError("You provided "+(e!==null&&typeof e=="object"?"an invalid object":"'"+e+"'")+" where a stream was expected. You can provide an Observable, Promise, ReadableStream, Array, AsyncIterable, or Iterable.")}function zi(){return typeof Symbol!="function"||!Symbol.iterator?"@@iterator":Symbol.iterator}var Yt=zi();function Gt(e){return C(e==null?void 0:e[Yt])}function Bt(e){return un(this,arguments,function(){var r,n,o,i;return $t(this,function(s){switch(s.label){case 0:r=e.getReader(),s.label=1;case 1:s.trys.push([1,,9,10]),s.label=2;case 2:return[4,et(r.read())];case 3:return n=s.sent(),o=n.value,i=n.done,i?[4,et(void 0)]:[3,5];case 4:return[2,s.sent()];case 5:return[4,et(o)];case 6:return[4,s.sent()];case 7:return s.sent(),[3,2];case 8:return[3,10];case 9:return r.releaseLock(),[7];case 10:return[2]}})})}function Jt(e){return C(e==null?void 0:e.getReader)}function U(e){if(e instanceof F)return e;if(e!=null){if(qt(e))return Ni(e);if(pt(e))return qi(e);if(Nt(e))return Ki(e);if(Kt(e))return On(e);if(Gt(e))return Qi(e);if(Jt(e))return Yi(e)}throw Qt(e)}function Ni(e){return new F(function(t){var r=e[ft]();if(C(r.subscribe))return r.subscribe(t);throw new TypeError("Provided object does not correctly implement Symbol.observable")})}function qi(e){return new F(function(t){for(var r=0;r=2;return function(n){return n.pipe(e?A(function(o,i){return e(o,i,n)}):de,ge(1),r?He(t):Dn(function(){return new Zt}))}}function Vn(){for(var e=[],t=0;t=2,!0))}function pe(e){e===void 0&&(e={});var t=e.connector,r=t===void 0?function(){return new x}:t,n=e.resetOnError,o=n===void 0?!0:n,i=e.resetOnComplete,s=i===void 0?!0:i,a=e.resetOnRefCountZero,f=a===void 0?!0:a;return function(c){var u,p,m,d=0,h=!1,v=!1,Q=function(){p==null||p.unsubscribe(),p=void 0},B=function(){Q(),u=m=void 0,h=v=!1},N=function(){var O=u;B(),O==null||O.unsubscribe()};return y(function(O,Qe){d++,!v&&!h&&Q();var De=m=m!=null?m:r();Qe.add(function(){d--,d===0&&!v&&!h&&(p=$r(N,f))}),De.subscribe(Qe),!u&&d>0&&(u=new rt({next:function($e){return De.next($e)},error:function($e){v=!0,Q(),p=$r(B,o,$e),De.error($e)},complete:function(){h=!0,Q(),p=$r(B,s),De.complete()}}),U(O).subscribe(u))})(c)}}function $r(e,t){for(var r=[],n=2;ne.next(document)),e}function K(e,t=document){return Array.from(t.querySelectorAll(e))}function z(e,t=document){let r=ce(e,t);if(typeof r=="undefined")throw new ReferenceError(`Missing element: expected "${e}" to be present`);return r}function ce(e,t=document){return t.querySelector(e)||void 0}function _e(){return document.activeElement instanceof HTMLElement&&document.activeElement||void 0}function tr(e){return L(b(document.body,"focusin"),b(document.body,"focusout")).pipe(ke(1),l(()=>{let t=_e();return typeof t!="undefined"?e.contains(t):!1}),V(e===_e()),J())}function Xe(e){return{x:e.offsetLeft,y:e.offsetTop}}function Kn(e){return L(b(window,"load"),b(window,"resize")).pipe(Ce(0,Oe),l(()=>Xe(e)),V(Xe(e)))}function rr(e){return{x:e.scrollLeft,y:e.scrollTop}}function dt(e){return L(b(e,"scroll"),b(window,"resize")).pipe(Ce(0,Oe),l(()=>rr(e)),V(rr(e)))}var Yn=function(){if(typeof Map!="undefined")return Map;function e(t,r){var n=-1;return t.some(function(o,i){return o[0]===r?(n=i,!0):!1}),n}return function(){function t(){this.__entries__=[]}return Object.defineProperty(t.prototype,"size",{get:function(){return this.__entries__.length},enumerable:!0,configurable:!0}),t.prototype.get=function(r){var n=e(this.__entries__,r),o=this.__entries__[n];return o&&o[1]},t.prototype.set=function(r,n){var o=e(this.__entries__,r);~o?this.__entries__[o][1]=n:this.__entries__.push([r,n])},t.prototype.delete=function(r){var n=this.__entries__,o=e(n,r);~o&&n.splice(o,1)},t.prototype.has=function(r){return!!~e(this.__entries__,r)},t.prototype.clear=function(){this.__entries__.splice(0)},t.prototype.forEach=function(r,n){n===void 0&&(n=null);for(var o=0,i=this.__entries__;o0},e.prototype.connect_=function(){!Wr||this.connected_||(document.addEventListener("transitionend",this.onTransitionEnd_),window.addEventListener("resize",this.refresh),va?(this.mutationsObserver_=new MutationObserver(this.refresh),this.mutationsObserver_.observe(document,{attributes:!0,childList:!0,characterData:!0,subtree:!0})):(document.addEventListener("DOMSubtreeModified",this.refresh),this.mutationEventsAdded_=!0),this.connected_=!0)},e.prototype.disconnect_=function(){!Wr||!this.connected_||(document.removeEventListener("transitionend",this.onTransitionEnd_),window.removeEventListener("resize",this.refresh),this.mutationsObserver_&&this.mutationsObserver_.disconnect(),this.mutationEventsAdded_&&document.removeEventListener("DOMSubtreeModified",this.refresh),this.mutationsObserver_=null,this.mutationEventsAdded_=!1,this.connected_=!1)},e.prototype.onTransitionEnd_=function(t){var r=t.propertyName,n=r===void 0?"":r,o=ba.some(function(i){return!!~n.indexOf(i)});o&&this.refresh()},e.getInstance=function(){return this.instance_||(this.instance_=new e),this.instance_},e.instance_=null,e}(),Gn=function(e,t){for(var r=0,n=Object.keys(t);r0},e}(),Jn=typeof WeakMap!="undefined"?new WeakMap:new Yn,Xn=function(){function e(t){if(!(this instanceof e))throw new TypeError("Cannot call a class as a function.");if(!arguments.length)throw new TypeError("1 argument required, but only 0 present.");var r=ga.getInstance(),n=new La(t,r,this);Jn.set(this,n)}return e}();["observe","unobserve","disconnect"].forEach(function(e){Xn.prototype[e]=function(){var t;return(t=Jn.get(this))[e].apply(t,arguments)}});var Aa=function(){return typeof nr.ResizeObserver!="undefined"?nr.ResizeObserver:Xn}(),Zn=Aa;var eo=new x,Ca=$(()=>k(new Zn(e=>{for(let t of e)eo.next(t)}))).pipe(g(e=>L(ze,k(e)).pipe(R(()=>e.disconnect()))),X(1));function he(e){return{width:e.offsetWidth,height:e.offsetHeight}}function ye(e){return Ca.pipe(S(t=>t.observe(e)),g(t=>eo.pipe(A(({target:r})=>r===e),R(()=>t.unobserve(e)),l(()=>he(e)))),V(he(e)))}function bt(e){return{width:e.scrollWidth,height:e.scrollHeight}}function ar(e){let t=e.parentElement;for(;t&&(e.scrollWidth<=t.scrollWidth&&e.scrollHeight<=t.scrollHeight);)t=(e=t).parentElement;return t?e:void 0}var to=new x,Ra=$(()=>k(new IntersectionObserver(e=>{for(let t of e)to.next(t)},{threshold:0}))).pipe(g(e=>L(ze,k(e)).pipe(R(()=>e.disconnect()))),X(1));function sr(e){return Ra.pipe(S(t=>t.observe(e)),g(t=>to.pipe(A(({target:r})=>r===e),R(()=>t.unobserve(e)),l(({isIntersecting:r})=>r))))}function ro(e,t=16){return dt(e).pipe(l(({y:r})=>{let n=he(e),o=bt(e);return r>=o.height-n.height-t}),J())}var cr={drawer:z("[data-md-toggle=drawer]"),search:z("[data-md-toggle=search]")};function no(e){return cr[e].checked}function Ke(e,t){cr[e].checked!==t&&cr[e].click()}function Ue(e){let t=cr[e];return b(t,"change").pipe(l(()=>t.checked),V(t.checked))}function ka(e,t){switch(e.constructor){case HTMLInputElement:return e.type==="radio"?/^Arrow/.test(t):!0;case HTMLSelectElement:case HTMLTextAreaElement:return!0;default:return e.isContentEditable}}function Ha(){return L(b(window,"compositionstart").pipe(l(()=>!0)),b(window,"compositionend").pipe(l(()=>!1))).pipe(V(!1))}function oo(){let e=b(window,"keydown").pipe(A(t=>!(t.metaKey||t.ctrlKey)),l(t=>({mode:no("search")?"search":"global",type:t.key,claim(){t.preventDefault(),t.stopPropagation()}})),A(({mode:t,type:r})=>{if(t==="global"){let n=_e();if(typeof n!="undefined")return!ka(n,r)}return!0}),pe());return Ha().pipe(g(t=>t?_:e))}function le(){return new URL(location.href)}function ot(e){location.href=e.href}function io(){return new x}function ao(e,t){if(typeof t=="string"||typeof t=="number")e.innerHTML+=t.toString();else if(t instanceof Node)e.appendChild(t);else if(Array.isArray(t))for(let r of t)ao(e,r)}function M(e,t,...r){let n=document.createElement(e);if(t)for(let o of Object.keys(t))typeof t[o]!="undefined"&&(typeof t[o]!="boolean"?n.setAttribute(o,t[o]):n.setAttribute(o,""));for(let o of r)ao(n,o);return n}function fr(e){if(e>999){let t=+((e-950)%1e3>99);return`${((e+1e-6)/1e3).toFixed(t)}k`}else return e.toString()}function so(){return location.hash.substring(1)}function Dr(e){let t=M("a",{href:e});t.addEventListener("click",r=>r.stopPropagation()),t.click()}function Pa(e){return L(b(window,"hashchange"),e).pipe(l(so),V(so()),A(t=>t.length>0),X(1))}function co(e){return Pa(e).pipe(l(t=>ce(`[id="${t}"]`)),A(t=>typeof t!="undefined"))}function Vr(e){let t=matchMedia(e);return er(r=>t.addListener(()=>r(t.matches))).pipe(V(t.matches))}function fo(){let e=matchMedia("print");return L(b(window,"beforeprint").pipe(l(()=>!0)),b(window,"afterprint").pipe(l(()=>!1))).pipe(V(e.matches))}function zr(e,t){return e.pipe(g(r=>r?t():_))}function ur(e,t={credentials:"same-origin"}){return ue(fetch(`${e}`,t)).pipe(fe(()=>_),g(r=>r.status!==200?Ot(()=>new Error(r.statusText)):k(r)))}function We(e,t){return ur(e,t).pipe(g(r=>r.json()),X(1))}function uo(e,t){let r=new DOMParser;return ur(e,t).pipe(g(n=>n.text()),l(n=>r.parseFromString(n,"text/xml")),X(1))}function pr(e){let t=M("script",{src:e});return $(()=>(document.head.appendChild(t),L(b(t,"load"),b(t,"error").pipe(g(()=>Ot(()=>new ReferenceError(`Invalid script: ${e}`))))).pipe(l(()=>{}),R(()=>document.head.removeChild(t)),ge(1))))}function po(){return{x:Math.max(0,scrollX),y:Math.max(0,scrollY)}}function lo(){return L(b(window,"scroll",{passive:!0}),b(window,"resize",{passive:!0})).pipe(l(po),V(po()))}function mo(){return{width:innerWidth,height:innerHeight}}function ho(){return b(window,"resize",{passive:!0}).pipe(l(mo),V(mo()))}function bo(){return Y([lo(),ho()]).pipe(l(([e,t])=>({offset:e,size:t})),X(1))}function lr(e,{viewport$:t,header$:r}){let n=t.pipe(ee("size")),o=Y([n,r]).pipe(l(()=>Xe(e)));return Y([r,t,o]).pipe(l(([{height:i},{offset:s,size:a},{x:f,y:c}])=>({offset:{x:s.x-f,y:s.y-c+i},size:a})))}(()=>{function e(n,o){parent.postMessage(n,o||"*")}function t(...n){return n.reduce((o,i)=>o.then(()=>new Promise(s=>{let a=document.createElement("script");a.src=i,a.onload=s,document.body.appendChild(a)})),Promise.resolve())}var r=class extends EventTarget{constructor(n){super(),this.url=n,this.m=i=>{i.source===this.w&&(this.dispatchEvent(new MessageEvent("message",{data:i.data})),this.onmessage&&this.onmessage(i))},this.e=(i,s,a,f,c)=>{if(s===`${this.url}`){let u=new ErrorEvent("error",{message:i,filename:s,lineno:a,colno:f,error:c});this.dispatchEvent(u),this.onerror&&this.onerror(u)}};let o=document.createElement("iframe");o.hidden=!0,document.body.appendChild(this.iframe=o),this.w.document.open(),this.w.document.write(`

How to Use AVX512 in Golang via C Compiler

AVX512 is the latest generation of SIMD instructions released by Intel, which can process 512 bits of data in one instruction cycle, equivalent to 16 single-precision floating point numbers or 8 double-precision floating point numbers. The training and inference process of recommendation models in Gorse requires a lot of vector computation, and AVX512 can theoretically bring some acceleration effect. Unfortunately, the Go compiler does not automatically generate SIMD instructions.

MinIO had open-sourced a tool to convert Intel assembly to Go assembly c2goasm. First, the vectorized functions are implemented in C, and the assembly containing the SIMD instructions is compiled by Clang. Then, since Go assembly supports AVX512, the functions implemented by SIMD can be called through Go assembly. The c2goasm solution is very effective, however, the project has not been updated for almost 4 years, and after testing it cannot handle AVX512 instructions.

To use AVX512 in Go, we have developed a toolkit for compiling C into Go assembly functions goat and implemented a library of vectorized functions with the help of goat github.com/gorse-io/gorse/base/floats. Inheriting the idea of c2goasm, goat implements further enhancements.

  1. Starting directly from the C source code to get Go assembly functions, the user does not have to compile the C source code itself.
  2. It will also generate Go function definitions based on C function definitions, so users do not need to write Go function definitions by hand.

The post will detail the technical implementation of goat ideas, welcome to read the code github.com/gorse-io/gorse/cmd/goat.

flowchart LR
-  c[C source]-->intel[Intel assembly]
-  intel-->goasm[Go assembly]
-  c-->go[Go function definition]

Compile Assembly from C

The C implementation of the function _mm512_mul_to multiplies two floating point arrays and then saves the result in a third array. In general, the compiler automatically generates assembly that uses SIMD. Here, intrinsic is used to ensure that SIMD instructions are generated.

#include <immintrin.h>
-
-void _mm512_mul_to(float *a, float *b, float *c, int64_t n)
-{
-    int epoch = n / 16;
-    int remain = n % 16;
-    for (int i = 0; i < epoch; i++)
-    {
-        __m512 v1 = _mm512_loadu_ps(a);
-        __m512 v2 = _mm512_loadu_ps(b);
-        __m512 v = _mm512_mul_ps(v1, v2);
-        _mm512_storeu_ps(c, v);
-        a += 16;
-        b += 16;
-        c += 16;
-    }
-    if (remain >= 8)
-    {
-        __m256 v1 = _mm256_loadu_ps(a);
-        __m256 v2 = _mm256_loadu_ps(b);
-        __m256 v = _mm256_mul_ps(v1, v2);
-        _mm256_storeu_ps(c, v);
-        a += 8;
-        b += 8;
-        c += 8;
-        remain -= 8;
-    }
-    for (int i = 0; i < remain; i++)
-    {
-        c[i] = a[i] * b[i];
-    }
-}
-

Save the code to mm512_mul_to.c and use the following command to compile the C code to assembly as well as binary.

# Generate assembly
-clang -S -c mm512_mul_to.c -o mm512_mul_to.s \
-  -O3 -mavx -mfma -mavx512f -mavx512dq \
-  -mno-red-zone -mstackrealign -mllvm -inline-threshold=1000 \
-  -fno-asynchronous-unwind-tables -fno-exceptions -fno-rtti
-
-# Generate binary
-clang -c mm512_mul_to.c -o mm512_mul_to.o \
-  -O3 -mavx -mfma -mavx512f -mavx512dq \
-  -mno-red-zone -mstackrealign -mllvm -inline-threshold=1000 \
-  -fno-asynchronous-unwind-tables -fno-exceptions -fno-rtti
-

::: details mm512_mul_to.s

 .globl _mm512_mul_to                   # -- Begin function _mm512_mul_to
- .p2align 4, 0x90
- .type _mm512_mul_to,@function
-_mm512_mul_to:                          # @_mm512_mul_to
-# %bb.0:
- pushq %rbp
- movq %rsp, %rbp
- andq $-8, %rsp
- leaq 15(%rcx), %r9
- testq %rcx, %rcx
- cmovnsq %rcx, %r9
- shrq $4, %r9
- movl %r9d, %eax
- shll $4, %eax
- subl %eax, %ecx
- testl %r9d, %r9d
- jle .LBB3_6
-# %bb.1:
- leal -1(%r9), %eax
- movl %r9d, %r8d
- andl $3, %r8d
- cmpl $3, %eax
- jb .LBB3_4
-# %bb.2:
- andl $-4, %r9d
- negl %r9d
- .p2align 4, 0x90
-.LBB3_3:                                # =>This Inner Loop Header: Depth=1
- vmovups (%rdi), %zmm0
- vmulps (%rsi), %zmm0, %zmm0
- vmovups %zmm0, (%rdx)
- vmovups 64(%rdi), %zmm0
- vmulps 64(%rsi), %zmm0, %zmm0
- vmovups %zmm0, 64(%rdx)
- vmovups 128(%rdi), %zmm0
- vmulps 128(%rsi), %zmm0, %zmm0
- vmovups %zmm0, 128(%rdx)
- vmovups 192(%rdi), %zmm0
- vmulps 192(%rsi), %zmm0, %zmm0
- vmovups %zmm0, 192(%rdx)
- addq $256, %rdi                      # imm = 0x100
- addq $256, %rsi                      # imm = 0x100
- addq $256, %rdx                      # imm = 0x100
- addl $4, %r9d
- jne .LBB3_3
-.LBB3_4:
- testl %r8d, %r8d
- je .LBB3_6
- .p2align 4, 0x90
-.LBB3_5:                                # =>This Inner Loop Header: Depth=1
- vmovups (%rdi), %zmm0
- vmulps (%rsi), %zmm0, %zmm0
- vmovups %zmm0, (%rdx)
- addq $64, %rdi
- addq $64, %rsi
- addq $64, %rdx
- addl $-1, %r8d
- jne .LBB3_5
-.LBB3_6:
- cmpl $7, %ecx
- jle .LBB3_8
-# %bb.7:
- vmovups (%rdi), %ymm0
- vmulps (%rsi), %ymm0, %ymm0
- vmovups %ymm0, (%rdx)
- addq $32, %rdi
- addq $32, %rsi
- addq $32, %rdx
- addl $-8, %ecx
-.LBB3_8:
- testl %ecx, %ecx
- jle .LBB3_14
-# %bb.9:
- movl %ecx, %ecx
- leaq -1(%rcx), %rax
- movl %ecx, %r8d
- andl $3, %r8d
- cmpq $3, %rax
- jae .LBB3_15
-# %bb.10:
- xorl %eax, %eax
- jmp .LBB3_11
-.LBB3_15:
- andl $-4, %ecx
- xorl %eax, %eax
- .p2align 4, 0x90
-.LBB3_16:                               # =>This Inner Loop Header: Depth=1
- vmovss (%rdi,%rax,4), %xmm0            # xmm0 = mem[0],zero,zero,zero
- vmulss (%rsi,%rax,4), %xmm0, %xmm0
- vmovss %xmm0, (%rdx,%rax,4)
- vmovss 4(%rdi,%rax,4), %xmm0           # xmm0 = mem[0],zero,zero,zero
- vmulss 4(%rsi,%rax,4), %xmm0, %xmm0
- vmovss %xmm0, 4(%rdx,%rax,4)
- vmovss 8(%rdi,%rax,4), %xmm0           # xmm0 = mem[0],zero,zero,zero
- vmulss 8(%rsi,%rax,4), %xmm0, %xmm0
- vmovss %xmm0, 8(%rdx,%rax,4)
- vmovss 12(%rdi,%rax,4), %xmm0          # xmm0 = mem[0],zero,zero,zero
- vmulss 12(%rsi,%rax,4), %xmm0, %xmm0
- vmovss %xmm0, 12(%rdx,%rax,4)
- addq $4, %rax
- cmpq %rax, %rcx
- jne .LBB3_16
-.LBB3_11:
- testq %r8, %r8
- je .LBB3_14
-# %bb.12:
- leaq (%rdx,%rax,4), %rcx
- leaq (%rsi,%rax,4), %rdx
- leaq (%rdi,%rax,4), %rax
- xorl %esi, %esi
- .p2align 4, 0x90
-.LBB3_13:                               # =>This Inner Loop Header: Depth=1
- vmovss (%rax,%rsi,4), %xmm0            # xmm0 = mem[0],zero,zero,zero
- vmulss (%rdx,%rsi,4), %xmm0, %xmm0
- vmovss %xmm0, (%rcx,%rsi,4)
- addq $1, %rsi
- cmpq %rsi, %r8
- jne .LBB3_13
-.LBB3_14:
- movq %rbp, %rsp
- popq %rbp
- vzeroupper
- retq
-.Lfunc_end3:
- .size _mm512_mul_to, .Lfunc_end3-_mm512_mul_to
-                                        # -- End function
-

:::

In Go, it is easy to execute the compile command with exec.Command.

Construct Go Code from Assembly

Convert Assembly

mm512_mul_to.o is the binary of the function after it has been compiled. By using objdump you can see that each assembly has been converted to machine code.

objdump -d mm512_mul_to.o --insn-width 16
-

::: details objdump output

0000000000000460 <_mm512_mul_to>:
- 460:   55                              push   %rbp
- 461:   48 89 e5                        mov    %rsp,%rbp
- 464:   48 83 e4 f8                     and    $0xfffffffffffffff8,%rsp
- 468:   4c 8d 49 0f                     lea    0xf(%rcx),%r9
- 46c:   48 85 c9                        test   %rcx,%rcx
- 46f:   4c 0f 49 c9                     cmovns %rcx,%r9
- 473:   49 c1 e9 04                     shr    $0x4,%r9
- 477:   44 89 c8                        mov    %r9d,%eax
- 47a:   c1 e0 04                        shl    $0x4,%eax
- 47d:   29 c1                           sub    %eax,%ecx
- 47f:   45 85 c9                        test   %r9d,%r9d
- 482:   0f 8e bc 00 00 00               jle    544 <_mm512_mul_to+0xe4>
- 488:   41 8d 41 ff                     lea    -0x1(%r9),%eax
- 48c:   45 89 c8                        mov    %r9d,%r8d
- 48f:   41 83 e0 03                     and    $0x3,%r8d
- 493:   83 f8 03                        cmp    $0x3,%eax
- 496:   72 74                           jb     50c <_mm512_mul_to+0xac>
- 498:   41 83 e1 fc                     and    $0xfffffffc,%r9d
- 49c:   41 f7 d9                        neg    %r9d
- 49f:   90                              nop
- 4a0:   62 f1 7c 48 10 07               vmovups (%rdi),%zmm0
- 4a6:   62 f1 7c 48 59 06               vmulps (%rsi),%zmm0,%zmm0
- 4ac:   62 f1 7c 48 11 02               vmovups %zmm0,(%rdx)
- 4b2:   62 f1 7c 48 10 47 01            vmovups 0x40(%rdi),%zmm0
- 4b9:   62 f1 7c 48 59 46 01            vmulps 0x40(%rsi),%zmm0,%zmm0
- 4c0:   62 f1 7c 48 11 42 01            vmovups %zmm0,0x40(%rdx)
- 4c7:   62 f1 7c 48 10 47 02            vmovups 0x80(%rdi),%zmm0
- 4ce:   62 f1 7c 48 59 46 02            vmulps 0x80(%rsi),%zmm0,%zmm0
- 4d5:   62 f1 7c 48 11 42 02            vmovups %zmm0,0x80(%rdx)
- 4dc:   62 f1 7c 48 10 47 03            vmovups 0xc0(%rdi),%zmm0
- 4e3:   62 f1 7c 48 59 46 03            vmulps 0xc0(%rsi),%zmm0,%zmm0
- 4ea:   62 f1 7c 48 11 42 03            vmovups %zmm0,0xc0(%rdx)
- 4f1:   48 81 c7 00 01 00 00            add    $0x100,%rdi
- 4f8:   48 81 c6 00 01 00 00            add    $0x100,%rsi
- 4ff:   48 81 c2 00 01 00 00            add    $0x100,%rdx
- 506:   41 83 c1 04                     add    $0x4,%r9d
- 50a:   75 94                           jne    4a0 <_mm512_mul_to+0x40>
- 50c:   45 85 c0                        test   %r8d,%r8d
- 50f:   74 33                           je     544 <_mm512_mul_to+0xe4>
- 511:   66 2e 0f 1f 84 00 00 00 00 00   nopw   %cs:0x0(%rax,%rax,1)
- 51b:   0f 1f 44 00 00                  nopl   0x0(%rax,%rax,1)
- 520:   62 f1 7c 48 10 07               vmovups (%rdi),%zmm0
- 526:   62 f1 7c 48 59 06               vmulps (%rsi),%zmm0,%zmm0
- 52c:   62 f1 7c 48 11 02               vmovups %zmm0,(%rdx)
- 532:   48 83 c7 40                     add    $0x40,%rdi
- 536:   48 83 c6 40                     add    $0x40,%rsi
- 53a:   48 83 c2 40                     add    $0x40,%rdx
- 53e:   41 83 c0 ff                     add    $0xffffffff,%r8d
- 542:   75 dc                           jne    520 <_mm512_mul_to+0xc0>
- 544:   83 f9 07                        cmp    $0x7,%ecx
- 547:   7e 1b                           jle    564 <_mm512_mul_to+0x104>
- 549:   c5 fc 10 07                     vmovups (%rdi),%ymm0
- 54d:   c5 fc 59 06                     vmulps (%rsi),%ymm0,%ymm0
- 551:   c5 fc 11 02                     vmovups %ymm0,(%rdx)
- 555:   48 83 c7 20                     add    $0x20,%rdi
- 559:   48 83 c6 20                     add    $0x20,%rsi
- 55d:   48 83 c2 20                     add    $0x20,%rdx
- 561:   83 c1 f8                        add    $0xfffffff8,%ecx
- 564:   85 c9                           test   %ecx,%ecx
- 566:   0f 8e ac 00 00 00               jle    618 <_mm512_mul_to+0x1b8>
- 56c:   89 c9                           mov    %ecx,%ecx
- 56e:   48 8d 41 ff                     lea    -0x1(%rcx),%rax
- 572:   41 89 c8                        mov    %ecx,%r8d
- 575:   41 83 e0 03                     and    $0x3,%r8d
- 579:   48 83 f8 03                     cmp    $0x3,%rax
- 57d:   73 04                           jae    583 <_mm512_mul_to+0x123>
- 57f:   31 c0                           xor    %eax,%eax
- 581:   eb 5b                           jmp    5de <_mm512_mul_to+0x17e>
- 583:   83 e1 fc                        and    $0xfffffffc,%ecx
- 586:   31 c0                           xor    %eax,%eax
- 588:   0f 1f 84 00 00 00 00 00         nopl   0x0(%rax,%rax,1)
- 590:   c5 fa 10 04 87                  vmovss (%rdi,%rax,4),%xmm0
- 595:   c5 fa 59 04 86                  vmulss (%rsi,%rax,4),%xmm0,%xmm0
- 59a:   c5 fa 11 04 82                  vmovss %xmm0,(%rdx,%rax,4)
- 59f:   c5 fa 10 44 87 04               vmovss 0x4(%rdi,%rax,4),%xmm0
- 5a5:   c5 fa 59 44 86 04               vmulss 0x4(%rsi,%rax,4),%xmm0,%xmm0
- 5ab:   c5 fa 11 44 82 04               vmovss %xmm0,0x4(%rdx,%rax,4)
- 5b1:   c5 fa 10 44 87 08               vmovss 0x8(%rdi,%rax,4),%xmm0
- 5b7:   c5 fa 59 44 86 08               vmulss 0x8(%rsi,%rax,4),%xmm0,%xmm0
- 5bd:   c5 fa 11 44 82 08               vmovss %xmm0,0x8(%rdx,%rax,4)
- 5c3:   c5 fa 10 44 87 0c               vmovss 0xc(%rdi,%rax,4),%xmm0
- 5c9:   c5 fa 59 44 86 0c               vmulss 0xc(%rsi,%rax,4),%xmm0,%xmm0
- 5cf:   c5 fa 11 44 82 0c               vmovss %xmm0,0xc(%rdx,%rax,4)
- 5d5:   48 83 c0 04                     add    $0x4,%rax
- 5d9:   48 39 c1                        cmp    %rax,%rcx
- 5dc:   75 b2                           jne    590 <_mm512_mul_to+0x130>
- 5de:   4d 85 c0                        test   %r8,%r8
- 5e1:   74 35                           je     618 <_mm512_mul_to+0x1b8>
- 5e3:   48 8d 0c 82                     lea    (%rdx,%rax,4),%rcx
- 5e7:   48 8d 14 86                     lea    (%rsi,%rax,4),%rdx
- 5eb:   48 8d 04 87                     lea    (%rdi,%rax,4),%rax
- 5ef:   31 f6                           xor    %esi,%esi
- 5f1:   66 2e 0f 1f 84 00 00 00 00 00   nopw   %cs:0x0(%rax,%rax,1)
- 5fb:   0f 1f 44 00 00                  nopl   0x0(%rax,%rax,1)
- 600:   c5 fa 10 04 b0                  vmovss (%rax,%rsi,4),%xmm0
- 605:   c5 fa 59 04 b2                  vmulss (%rdx,%rsi,4),%xmm0,%xmm0
- 60a:   c5 fa 11 04 b1                  vmovss %xmm0,(%rcx,%rsi,4)
- 60f:   48 83 c6 01                     add    $0x1,%rsi
- 613:   49 39 f0                        cmp    %rsi,%r8
- 616:   75 e8                           jne    600 <_mm512_mul_to+0x1a0>
- 618:   48 89 ec                        mov    %rbp,%rsp
- 61b:   5d                              pop    %rbp
- 61c:   c5 f8 77                        vzeroupper 
- 61f:   c3                              retq   
-

:::

Encoding Machine Code

Go assembly provides three instructions for representing binary machine code.

  • BYTE encodes one byte of binary data
  • WORD encodes two bytes of binary data
  • LONG encodes four bytes of binary data

If the instruction machine code length is exactly a multiple of two, for example

 498: 41 83 e1 fc and $0xfffffffc,%r9d
-

can be converted to

 LONG $0xfce18341
-

But if the length is not a multiple of two, for example

 4b2: 62 f1 7c 48 10 47 01 vmovups 0x40(%rdi),%zmm0
-

It requires a combination of three instructions to represent

 LONG $0x487cf162; WORD $0x4710; BYTE $0x01 // vmovups  64(%rdi), %zmm0
-

Note that the byte order of the instruction encoding and the byte order of the objdump output are reversed.

Function Definition and Parameters

In C assembly, if the function has no more than 6 arguments, the arguments are stored in registers and passed to the function in the following order of placement: %rdi, %rsi, %rdx, %rcx, %r8 and %r9. However, in Go assembly, function arguments are placed in memory starting from the address held in the FP register, which requires us to move the arguments in memory to the register.

The _mm512_mul_to function has four arguments, so it is necessary to move the four arguments before the function starts.

TEXT -_mm512_mul_to(SB), $0-32
-  MOVQ a+0(FP), DI
-  MOVQ b+8(FP), SI
-  MOVQ c+16(FP), DX
-  MOVQ n+24(FP), CX
-

The function definition consists of three parts: the TEXT keyword, the name starting with - and ending with (SB), and finally the parameter memory size of 32 bytes. The information about the number of arguments is not available in assembly and needs to be obtained from the C function definition (refer to Generating go function definitions).

Redirect Jump Instruction

x86 jump instructions jump to absolute addresses and direct encoding of jump instructions does not work. Therefore, jump instructions need to be converted to jump instructions in Go assembly.

  • Converting tags: Go assembly tags cannot start with ., so you need to remove the ..
  • Conversion commands: Go assembly jump commands are in uppercase.

After the above three steps, the final Go assembly code is obtained

::: details Go assembly

TEXT -_mm512_mul_to(SB), $0-32
- MOVQ a+0(FP), DI
- MOVQ b+8(FP), SI
- MOVQ c+16(FP), DX
- MOVQ n+24(FP), CX
- BYTE $0x55 // pushq %rbp
- WORD $0x8948; BYTE $0xe5 // movq %rsp, %rbp
- LONG $0xf8e48348 // andq $-8, %rsp
- LONG $0x0f498d4c // leaq 15(%rcx), %r9
- WORD $0x8548; BYTE $0xc9 // testq %rcx, %rcx
- LONG $0xc9490f4c // cmovnsq %rcx, %r9
- LONG $0x04e9c149 // shrq $4, %r9
- WORD $0x8944; BYTE $0xc8 // movl %r9d, %eax
- WORD $0xe0c1; BYTE $0x04 // shll $4, %eax
- WORD $0xc129 // subl %eax, %ecx
- WORD $0x8545; BYTE $0xc9 // testl %r9d, %r9d
- JLE LBB3_6
- LONG $0xff418d41 // leal -1(%r9), %eax
- WORD $0x8945; BYTE $0xc8 // movl %r9d, %r8d
- LONG $0x03e08341 // andl $3, %r8d
- WORD $0xf883; BYTE $0x03 // cmpl $3, %eax
- JB LBB3_4
- LONG $0xfce18341 // andl $-4, %r9d
- WORD $0xf741; BYTE $0xd9 // negl %r9d
-
-LBB3_3:
- LONG $0x487cf162; WORD $0x0710 // vmovups (%rdi), %zmm0
- LONG $0x487cf162; WORD $0x0659 // vmulps (%rsi), %zmm0, %zmm0
- LONG $0x487cf162; WORD $0x0211 // vmovups %zmm0, (%rdx)
- LONG $0x487cf162; WORD $0x4710; BYTE $0x01 // vmovups 64(%rdi), %zmm0
- LONG $0x487cf162; WORD $0x4659; BYTE $0x01 // vmulps 64(%rsi), %zmm0, %zmm0
- LONG $0x487cf162; WORD $0x4211; BYTE $0x01 // vmovups %zmm0, 64(%rdx)
- LONG $0x487cf162; WORD $0x4710; BYTE $0x02 // vmovups 128(%rdi), %zmm0
- LONG $0x487cf162; WORD $0x4659; BYTE $0x02 // vmulps 128(%rsi), %zmm0, %zmm0
- LONG $0x487cf162; WORD $0x4211; BYTE $0x02 // vmovups %zmm0, 128(%rdx)
- LONG $0x487cf162; WORD $0x4710; BYTE $0x03 // vmovups 192(%rdi), %zmm0
- LONG $0x487cf162; WORD $0x4659; BYTE $0x03 // vmulps 192(%rsi), %zmm0, %zmm0
- LONG $0x487cf162; WORD $0x4211; BYTE $0x03 // vmovups %zmm0, 192(%rdx)
- LONG $0x00c78148; WORD $0x0001; BYTE $0x00 // addq $256, %rdi
- LONG $0x00c68148; WORD $0x0001; BYTE $0x00 // addq $256, %rsi
- LONG $0x00c28148; WORD $0x0001; BYTE $0x00 // addq $256, %rdx
- LONG $0x04c18341 // addl $4, %r9d
- JNE LBB3_3
-
-LBB3_4:
- WORD $0x8545; BYTE $0xc0 // testl %r8d, %r8d
- JE LBB3_6
-
-LBB3_5:
- LONG $0x487cf162; WORD $0x0710 // vmovups (%rdi), %zmm0
- LONG $0x487cf162; WORD $0x0659 // vmulps (%rsi), %zmm0, %zmm0
- LONG $0x487cf162; WORD $0x0211 // vmovups %zmm0, (%rdx)
- LONG $0x40c78348 // addq $64, %rdi
- LONG $0x40c68348 // addq $64, %rsi
- LONG $0x40c28348 // addq $64, %rdx
- LONG $0xffc08341 // addl $-1, %r8d
- JNE LBB3_5
-
-LBB3_6:
- WORD $0xf983; BYTE $0x07 // cmpl $7, %ecx
- JLE LBB3_8
- LONG $0x0710fcc5 // vmovups (%rdi), %ymm0
- LONG $0x0659fcc5 // vmulps (%rsi), %ymm0, %ymm0
- LONG $0x0211fcc5 // vmovups %ymm0, (%rdx)
- LONG $0x20c78348 // addq $32, %rdi
- LONG $0x20c68348 // addq $32, %rsi
- LONG $0x20c28348 // addq $32, %rdx
- WORD $0xc183; BYTE $0xf8 // addl $-8, %ecx
-
-LBB3_8:
- WORD $0xc985 // testl %ecx, %ecx
- JLE LBB3_14
- WORD $0xc989 // movl %ecx, %ecx
- LONG $0xff418d48 // leaq -1(%rcx), %rax
- WORD $0x8941; BYTE $0xc8 // movl %ecx, %r8d
- LONG $0x03e08341 // andl $3, %r8d
- LONG $0x03f88348 // cmpq $3, %rax
- JAE LBB3_15
- WORD $0xc031 // xorl %eax, %eax
- JMP LBB3_11
-
-LBB3_15:
- WORD $0xe183; BYTE $0xfc // andl $-4, %ecx
- WORD $0xc031 // xorl %eax, %eax
-
-LBB3_16:
- LONG $0x0410fac5; BYTE $0x87 // vmovss (%rdi,%rax,4), %xmm0
- LONG $0x0459fac5; BYTE $0x86 // vmulss (%rsi,%rax,4), %xmm0, %xmm0
- LONG $0x0411fac5; BYTE $0x82 // vmovss %xmm0, (%rdx,%rax,4)
- LONG $0x4410fac5; WORD $0x0487 // vmovss 4(%rdi,%rax,4), %xmm0
- LONG $0x4459fac5; WORD $0x0486 // vmulss 4(%rsi,%rax,4), %xmm0, %xmm0
- LONG $0x4411fac5; WORD $0x0482 // vmovss %xmm0, 4(%rdx,%rax,4)
- LONG $0x4410fac5; WORD $0x0887 // vmovss 8(%rdi,%rax,4), %xmm0
- LONG $0x4459fac5; WORD $0x0886 // vmulss 8(%rsi,%rax,4), %xmm0, %xmm0
- LONG $0x4411fac5; WORD $0x0882 // vmovss %xmm0, 8(%rdx,%rax,4)
- LONG $0x4410fac5; WORD $0x0c87 // vmovss 12(%rdi,%rax,4), %xmm0
- LONG $0x4459fac5; WORD $0x0c86 // vmulss 12(%rsi,%rax,4), %xmm0, %xmm0
- LONG $0x4411fac5; WORD $0x0c82 // vmovss %xmm0, 12(%rdx,%rax,4)
- LONG $0x04c08348 // addq $4, %rax
- WORD $0x3948; BYTE $0xc1 // cmpq %rax, %rcx
- JNE LBB3_16
-
-LBB3_11:
- WORD $0x854d; BYTE $0xc0 // testq %r8, %r8
- JE LBB3_14
- LONG $0x820c8d48 // leaq (%rdx,%rax,4), %rcx
- LONG $0x86148d48 // leaq (%rsi,%rax,4), %rdx
- LONG $0x87048d48 // leaq (%rdi,%rax,4), %rax
- WORD $0xf631 // xorl %esi, %esi
-
-LBB3_13:
- LONG $0x0410fac5; BYTE $0xb0 // vmovss (%rax,%rsi,4), %xmm0
- LONG $0x0459fac5; BYTE $0xb2 // vmulss (%rdx,%rsi,4), %xmm0, %xmm0
- LONG $0x0411fac5; BYTE $0xb1 // vmovss %xmm0, (%rcx,%rsi,4)
- LONG $0x01c68348 // addq $1, %rsi
- WORD $0x3949; BYTE $0xf0 // cmpq %rsi, %r8
- JNE LBB3_13
-
-LBB3_14:
- WORD $0x8948; BYTE $0xec // movq %rbp, %rsp
- BYTE $0x5d // popq %rbp
- WORD $0xf8c5; BYTE $0x77 // vzeroupper
- BYTE $0xc3 // retq
-

:::

Generate Go Function Definitions

Converting C function definitions to Go functions requires a C parser. cznic implements a C-to-Go converter in Go cc/v3 which provides a C parser. The function definition conversion consists of two parts.

  • Function name: Directly use the function name of the C function as the Go function name
  • Function arguments: Need to check that function arguments must be of the 64-bit type, passed in Go functions using unsafe.Pointer

The definition of the _mm512_mul_to conversion to a Go function is as follows

import "unsafe"
-
-//go:noescape
-func _mm512_mul_to(a, b, c, n unsafe.Pointer)
-

Build with go generate

For generated code, the go generate command can run custom commands to do conversions from C functions to Go functions. gorse added the go generate command to the floats_amd64.go file, which executes go generate . /... to automatically generate Go vectorized functions.

//go:generate go run ... /... /cmd/goat src/floats_avx.c -O3 -mavx
-//go:generate go run ... /... /cmd/goat src/floats_avx512.c -O3 -mavx -mfma -mavx512f -mavx512dq
-

Function Calls

Calling the vectorized function is not a simple task, because only new CPUs support AVX512, and some older CPUs do not even support AVX2, which requires the following steps when calling the vectorized function.

  1. Provide a non-vectorized function implementation for CPUs that do not support AVX512.
  2. Detect in the init function whether the currently running CPU supports AVX512.
  3. Wrapping an outer function automatically selects a vectorized or non-vectorized function depending on CPU instruction support.

The complete wrapper is as follows

import (
- "github.com/klauspost/cpuid/v2"
- "unsafe"
-)
-
-// CPU instruction flag
-var impl = Default
-
-const (
- Default int = iota
- AVX512
-)
-
-func init() {
- // Check if the CPU supports AVX512
- if cpuid.CPU.Supports(cpuid.AVX512F, cpuid.AVX512DQ) {
-  impl = AVX512
- }
-}
-
-// Non-vectorized implementation
-func mulTo(a, b, c []float32) {
- for i := range a {
-  c[i] = a[i] * b[i]
- }
-}
-
-// Wrapper functions
-func MulTo(a, b, c []float32) {
- if len(a) ! = len(b) || len(a) ! = len(c) {
-  panic("floats: slice lengths do not match")
- }
- // Automatic selection of functions for execution
- switch impl {
- case AVX512:
-  _mm512_mul_to(unsafe.Pointer(&a[0]), unsafe.Pointer(&b[0]), unsafe.Pointer(&c[0]), unsafe.Pointer(uintptr(len(a))))
- default:
-  mulTo(a, b, c)
- }
-}
-

::: note

Go assembly should be saved in a file with the .s extension and the Go definition and wrapper should be saved in Go files. They must be in the same directory with the same package name.

:::

Summary

Finally, we benchmark the performance of the non-vectorized function and the vectorized function.

  • Vectorized functions take significantly less time than non-vectorized functions, especially when the vector length is 128.
  • The AVX512 implementation has a slight performance increase relative to the AVX2 implementation.

::: echarts Vectorized Implementation Performance Benchmark

{
-  "legend": {
-    "data": ["SISD", "AVX2", "AVX512"]
-  },
-  "xAxis": {
-    "name": "vector-length",
-    "type": "category",
-    "data": [16, 32, 64, 128]
-  },
-  "yAxis": {
-    "name": "ns/op",
-    "type": "value"
-  },
-  "series": [
-    {
-      "name": "SISD",
-      "data": [10.56, 23.13, 38.01, 67.05],
-      "type": "bar",
-      "showBackground": true,
-      "backgroundStyle": {
-        "color": "rgba(180, 180, 180, 0.2)"
-      }
-    },
-    {
-      "name": "AVX2",
-      "data": [5.046, 5.138, 6.074, 8.294],
-      "type": "bar",
-      "showBackground": true,
-      "backgroundStyle": {
-        "color": "rgba(180, 180, 180, 0.2)"
-      }
-    },
-    {
-      "name": "AVX512",
-      "data": [4.741, 5.417, 5.583, 6.425],
-      "type": "bar",
-      "showBackground": true,
-      "backgroundStyle": {
-        "color": "rgba(180, 180, 180, 0.2)"
-      }
-    }
-  ]
-}
-

:::

In addition, the SIMD instructions Neon for ARM can also be used in Go by the idea of this post. goat also supports ARM, and Gorse also implements vectorized computation libraries for Neon instructions (refer to github.com/gorse-io/gorse/base/floats). You are welcome to use goat to build your own vectorized functions in Go projects, more questions are welcome in Issues or Discord.

Comments

\ No newline at end of file diff --git a/backup/blog/frontend-in-go-modules/index.html b/backup/blog/frontend-in-go-modules/index.html deleted file mode 100644 index 4b501ef..0000000 --- a/backup/blog/frontend-in-go-modules/index.html +++ /dev/null @@ -1,94 +0,0 @@ - Integrate Frontend Artifacts via Go Modules - CrossDB

Integrate Frontend Artifacts via Go Modules

The ability to compile into a single binary is a great feature of the Go programming language, avoiding dependency management at deployment time. However, if the project contains front-end code, we need to find a way to embed the front-end artifact into the Go binary at compile time. The compilation process is as follows.

flowchart LR
-  frontend[Compile frontend code]-->embed[Embed to Go code]-->build[Compile Go code]
  1. Compile the front-end code.
  2. Convert front-end artifacts to Go embedding files. For example, the project rakyll/statik is able to embed any file into the Go source code. In addition, the Go language currently provides an official embedding solution embed that allows embedding to be done during compilation, eliminating step 2.
  3. Build Go code.

Usually front-end code and Go code are placed in the same repository or put into the Go repository through git submodule and compiled through Makefile or build scripts. But such a solution has a small drawback, we need to install the front-end toolchain when developing a Go project. For a project like Gorse, most of the development work is on the backend, and the front-end changes are very minimal. It is unnecessary to compile the front-end every time the repository is pulled, so a solution is needed to avoid the front-end compilation process during Go development. The Gorse project compiles the dashboard front-end code into a separate Go package that is integrated into the main repository via Go modules.

Compile Front-end into Go Packages

rakyll/statik can embed front-end artifacts into Go code, then it is perfectly possible to have the converted code referenced by Gorse as a separate Go package. Then after each modification of the front-end code:

  1. Compile the front-end code.
  2. Use rakyll/statik to convert front-end artifacts to Go embedding files and initialize Go packages with go mod init and go mod tidy.
  3. Push the Go package to the repository or branch used to store the artifact.

The above process can be written as a script, and since the Gorse front-end repository is hosted on GitHub, it can be automatically updated each time the front-end code is committed through GitHub Action.

name: build
-
-on:
-  push:
-    branches: [ master ] # Triggered when master commits code
-
-jobs:
-  build:
-    runs-on: ubuntu-latest
-    steps:
-      # Compile front-end code
-      - uses: actions/checkout@v2
-      - name: Install dependencies
-        uses: borales/actions-yarn@v2.0.0
-        with:
-          cmd: install
-      - name: Build for production
-        uses: borales/actions-yarn@v2.0.0
-        with:
-          cmd: build
-      # Embed into Go source code
-      - name: Install Go
-        uses: actions/setup-go@v2
-      - name: Install statik & build embed files
-        run: |
-          export PATH=$PATH:$(go env GOPATH)/bin
-          go get github.com/rakyll/statik
-          statik -src=dist
-      # Commit Go source code
-      - name: Commit embed files
-        run: |
-          cd ...
-          git clone https://github.com/gorse-io/dashboard.git dashboard-statik
-          cd dashboard-statik
-          git config --local user.email "41898282+github-actions[bot]@users.noreply.github.com"
-          git config --local user.name "github-actions[bot]"
-          git checkout statik || git checkout --orphan statik
-          git rm -r --cached .
-          rm statik.go go.mod go.sum
-          cp -f ... /dashboard/statik/statik.go statik.go
-          go mod init github.com/gorse-io/dashboard
-          go mod tidy
-          git add statik.go go.mod go.sum
-          git commit -m "Build embed files"
-          git remote-set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}
-          git push origin statik
-

The above GitHub Action commits the Go code to the statik branch.

Using Front-end Artifacts via Go Modules

First, download the front-end artifact package. Since the product is stored on the statik branch, you need to add @statik after the package name.

go get -u github.com/gorse-io/dashboard@statik
-

Then, github.com/gorse-io/dashboard is imported and the front-end static files are saved in the filesystem created by fs.New().

import (
-  "github.com/rakyll/statik/fs"
-  _ "github.com/gorse-io/dashboard"
-)
-
-  // ...
-
-  statikFS, err := fs.New()
-  if err ! = nil {
-    log.Fatal(err)
-  }
-
-  // Example: Accessing the file system via HTTP
-  http.Handle("/", http.FileServer(statikFS))
-  http.ListenAndServe(":8080", nil)
-

Summary

Using Go modules to manage front-end artifacts is elegant, but only suitable for projects with few front-end changes. For projects where front-end and back-end development often needs joint debugging, such as mall projects, it is more appropriate to put the front-end and back-end code in the same repository.

Comments

\ No newline at end of file diff --git a/backup/features/index.html b/backup/features/index.html deleted file mode 100644 index 7906228..0000000 --- a/backup/features/index.html +++ /dev/null @@ -1 +0,0 @@ - Key Features - CrossDB

Key Features

High Performance

ACID Transactions

CrossDB's transactions support the ACID (Atomic, Consistent, Isolated and Durable) principles, which safeguard data integrity by guaranteeing that updates will complete together, or the database will roll back to a pre-transaction state.

Core RamDisk Database System Design

CrossDB was designed to be an application runtime DB from the beginning. An Ra database system eliminates disk and file I/O, cache management and other sources of latency. By working with data directly in main memory, eXtremeDB avoids the overhead of making multiple data copies and transfers inherent in disk-based DBMSs. Databases can be created in shared memory, enabling concurrent access by multiple processes. Read about the performance advantages of IMDS technology in one of our white papers.

In-Memory RamDisk or On-Disk Storage

CrossDB supports On-Disk Storage and In-Memory In addition to the core eXtremeDB in-memory database system, McObject’s eXtremeDB offers hybrid storage: on a table-by-table basis, tables can be designated for in-memory or on-disk storage (with flexible caching). Choose the best storage medium based on performance, persistence, cost and form factor.

Multi-Platform

eXtremeDB is available for use on all major workstation and server platforms: x32- and x64-bit versions of Windows and Linux, Linux on POWER8, Solaris Sparc, Solaris x86_64, HP-UX (Itanium), and AIX.

eXtremeDB is also available for all major embedded platforms: VxWorks, INTEGRITY, QNX, ThreadX, eCos, FreeRTOS and too many more to mention. Also x86, PowerPC, ARM, MIPS, etc.

eXtremeDB source code is available. The minimum requirement to build eXtremeDB for any platform is a 32-bit CPU and a decent quality C compiler.

B-Tree, R-Tree, Patricia Trie, KD-Tree, Trigram and Hash Indexes

eXtremeDB provides a wide range of database indexes, to boost application performance and minimize footprint. eXtremeDB offers R-trees for geospatial data, Patricia tries for IP/telecom, KD-trees for multi-dimensional data and Query-by-Example (QBE), B-trees, Trigram index for fuzzy search, hash indexes and more. For in-memory databases, rather than storing duplicate data, indexes contain only a reference to data, keeping memory requirements to an absolute minimum.

Wide Range of Supported Data Types

eXtremeDB supports a wide range of data types — including structures, arrays, boolean, binary, decimal, vectors and BLOBs — for maximum coding efficiency. Data can be stored in the same complex form in which it is used in the application, or as normalized relations. Learn more in our online documentation.

🌈 Single header and libray

CDB follows standard RDBMS model DB, Tabel, Record, Index, Cursor, Transaction, SQL.

🚊 C ORM

CDB supports C Struct to access DB direclty. Almost all C data types are supported Enum, Union, Bitfield, Array, nested struct, etc.

🚀 Trigger

CDB Supports Foreign Key, Trigger and Cascade Trigger. They can be used to do Data-driven programming and unit test.

⛽ PubSub

CDB supports PubSbub to subscribe DB from other source.

💮 RPC

CDB supports C struct serailization and a gRPC like RPC framework for C language in future.

🔱 Connector

CDB supports C/C++ and Python Connectors now, will support more connectors later.

♻️ Best Practice

Use CDB to Manage Data, do Data-Driven Programming, ISSU, distribut provisioning, etc.

🌄 Troubleshooting

CDB supports CLI, WEB, Staticstics, Status, Log, DB backup and restore to do troubleshooting.

\ No newline at end of file diff --git a/backup/index.html b/backup/index.html deleted file mode 100644 index bb6f42a..0000000 --- a/backup/index.html +++ /dev/null @@ -1,239 +0,0 @@ - Index - CrossDB

CrossDB
High Performance Embedded Database Development Framework✨

Start 🧭 Docs 💡 Download 💡

🌈 RDBMS

CDB follows standard RDBMS model DB, Tabel, Record, Index, Cursor, Transaction, SQL.

🚊 C ORM

CDB supports C Struct to access DB direclty. Almost all C data types are supported Enum, Union, Bitfield, Array, nested struct, etc.

🚀 Trigger

CDB Supports Foreign Key, Trigger and Cascade Trigger. They can be used to do Data-driven programming and unit test.

⛽ PubSub

CDB supports PubSbub to subscribe DB from other source.

💮 RPC

CDB supports C struct serailization and a gRPC like RPC framework for C language in future.

🔱 Connector

CDB supports C/C++ and Python Connectors now, will support more connectors later.

♻️ Best Practice

Use CDB to Manage Data, do Data-Driven Programming, ISSU, distribut provisioning, etc.

🌄 Troubleshooting

CDB supports CLI, WEB, Staticstics, Status, Log, DB backup and restore to do troubleshooting.

5 Minutes to Learn CrossDB

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
typedef struct {
-    uint32_t            prefix;
-    uint8_t             mask;
-    uint32_t            nexthop;
-    uint8_t             metric;
-    char                intf[16];
-    uint32_t            birth;
-    uint32_t            flags;
-} route_t;
-
-#undef  CROSS_STRUCT_NAME
-#define CROSS_STRUCT_NAME   route_t
-cross_field_t   route_schema[] = {
-    CROSS_FIELD (prefix,    UINT,   IPv4, 0),
-    CROSS_FIELD (mask,      UINT,   DFT,  0),
-    CROSS_FIELD (nexthop,   UINT,   IPv4, 0),
-    CROSS_FIELD (metric,    UINT,   DFT,  0),
-    CROSS_FIELD (type,      UINT,   DFT,  0),
-    CROSS_FIELD (intf,      CHAR,   DFT,  0),
-    CROSS_FIELD (birth,     UINT,   TS,   0),
-    CROSS_FIELD (flags,     UINT,   HEX,  0),
-    CROSS_END (route_t)
-};
-
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
// Open ondisk database (create if not exist)
-printf ("Open databse: mydb\n");
-ret = cross_dbOpen (&hDb, "mydb", CROSS_CREATE|CROSS_ONDISK);
-CHECK (ret, "Failed to open mydb");
-
-// Open table (create if not exist), PrimaryKey is ipAddr,mask
-printf ("Open table: route\n");
-ret = cross_tblOpen (hDb, &hRtTbl, "route", route_schema, "ipAddr,mask", CROSS_CREATE);
-CHECK (ret, "Failed to open route table");
-
-// Create index on nexthop to accelarate lookup by nexthop
-printf ("Create index: idx_nexthpp[\n");
-ret = cross_idxCreate (hRtTbl, "idx_nexthop", "nexthop", 0);
-CHECK (ret, "Failed to create index idx_nexthop");
-
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
printf ("Insert route 192.168.1.0/24->192.168.1.254\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.metric    = 1;
-route.flags     = 0;
-strcpy (route.intf, "eth1");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert row");
-
-printf ("Insert route 10.1.1.0/24->10.1.2.254\n");
-route.prefix    = IP4ADDR(100,1,0,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(10,1,1,254);
-route.metric    = 1;
-route.flags     = 0;
-strcpy (route.intf, "eth2");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert row");
-
-printf ("Insert route 10.1.2.0/24->10.1.2.254\n");
-route.prefix    = IP4ADDR(10,1,1,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.metric    = 2;
-route.flags     = 0;
-strcpy (route.intf, "eth2");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert row");
-
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
// Get all rows count
-count = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);
-printf ("Total %d routes\n", count);
-
-// Get rows count where nexthop=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-count = cross_dbGetRowsCount (hRtTbl, "nexthop", &route, 0);
-printf ("There're %d routes where nexthop=10.1.2.254\n", count);
-
-// Get single route 192.168.1.0/24 by PK
-memset (&route, 0, sizeof(route));
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbGetRowByPk (hRtTbl, &route, &route, 0); 
-CHECK (ret, "Failed to get row by PK");
-printf ("Get single route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n", 
-    IP4STR(route.prefix), route.mask, IP4STR(route.prefix), route.intf, route.metric, route.flags);
-
-// Use cursor to get routes where nexthop=10.1.2.254
-cross_cursor_h hCursor = NULL;
-route.nexthop   = IP4ADDR(10,1,2,254);
-count = cross_dbQueryRowsCursor (hRtTbl, &hCursor, "nexthop", &route, 0);
-CHECK (count, "Failed to query rows");
-printf ("Get %d routes where nexthop=10.1.2.254\n", count);
-while (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {
-    printf ("  route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n", 
-        IP4STR(route.prefix), route.mask, IP4STR(route.prefix), route.intf, route.metric, route.flags);
-}
-cross_cursorFree (hCursor);
-
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
printf ("Update single route 192.168.1.0/24 by PK: set flags 0->0x10\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 0x10;
-ret = cross_dbUpdRowByPk (hRtTbl, &route, "flags", &route, 0); 
-CHECK (ret, "Failed to update row by PK");
-
-// Update routes where nexthop=10.1.2.254: set flags 0->0x20
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.flags     = 0x20;
-count = cross_dbUpdateRows (hRtTbl, "nexthop", &route, "flags", &route, 0);
-CHECK (count, "Failed to update rows");
-printf ("Update %d routes where nexthop=10.1.2.254\n", count);  
-
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
printf ("Delete single route 192.168.1.0/24 by PK\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbDelRowByPk (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to delete row by PK");
-
-// Delete routes where nexthop=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-count = cross_dbDeleteRows (hRtTbl, "nexthop", &route, 0);
-CHECK (count, "Failed to update rows");
-printf ("Update %d routes where nexthop=10.1.2.254\n", count);
-

Sponsors

img

Note

Some text here

Some text here

Some text here

Addtional Content

Warning

Some text here

Some text here

Some text here

Addtional Content

\ No newline at end of file diff --git a/backup/introduction/index.html b/backup/introduction/index.html deleted file mode 100644 index 7c6d916..0000000 --- a/backup/introduction/index.html +++ /dev/null @@ -1 +0,0 @@ - Introduction - CrossDB

Introduction

CrossDB(CDB) is a powerful high-performance distrubuted embedded SQL RDBMS database. It's developed to improve development efficiency for embedded programmers.

  • You can use CDB RDBMS to Manage Program Data efficiently.
  • You can use CDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CDB RDBMS to refactor your code conveniently.
  • You can use CDB Index to optimize performance without changing your code.
  • You can use CDB Trigger to implement Data-Driven programming paradigm.
  • You can use CDB PUBSUB to subscribe DB from other process's DB either on same host or remote host.
  • You can use CDB to implement Centralize-DB programming paradigm.
  • You can use CDB eventloop to implement event-driven programming paradigm.
  • You can use CDB RPC to build distributed service.
  • You can use CDB CLI tool to debug running program's data in off-line way.
  • You can use CDB SQL to view/create/update/delete/filter program data.
  • You can use CDB Browser to view program data.
  • You can use CDB to do DB backup restore it.
  • You can use CDB DB Change Log to view DB change history with filter, backtrace, rate-limit, expiring, etc.
  • You can use Python connector to write unit test with SQL to test program.
  • You can copy the DB folders from device and open on PC/Server with cdb-cli or Python directly.
  • Use CDB DB-Driven Mode to Build Program Logic (Table Trigger, FK, auto delete, Cascade Trigger)
  • Use CDB Python Connector to do DB-Driven Unit Test
  • Use CDB Pub/Sub to Build Distributed System (Eventloop/Timer/WorkQueue, vs IPC, OOP [priv data])
  • Use CDB Pub/Sub to Build Centralized DB-Driven System (3rd lang, DM no checkpoint)
  • CDB Serialization and RPC
  • CDB SQL Connectors (RESP3, Python)
  • CDB SQL Drivers (C, Python)

Architecture

img

png relative img

Feature List

TDB

Support OS

  • Linux
  • Windows

Support Platform

Following platform are tested

  • x86
  • PowerPC
  • ARM

Porting

Log

Heart beat

Compiling

Linux

Build library

Build example

Windows

You can build with MinGW or MSYS2 or Cygwin

Build library MinGW MSYS2 Cygwin

Build example

\ No newline at end of file diff --git a/backup/reference/crossline/index.html b/backup/reference/crossline/index.html deleted file mode 100644 index dd2e190..0000000 --- a/backup/reference/crossline/index.html +++ /dev/null @@ -1,240 +0,0 @@ - Crossline - CrossDB

Crossline

I'm crossline

If you want to build a project, you can hire Scifabric for designing, building, and analysing your data. Check Scifabric solutions.

💡 Note: If you want to build a project, you can hire Scifabric for designing, building, and analysing your data. Check Scifabric solutions.

I'm developing a cross-platfrom command line tool which can support autocomplete and history. Readline library is the first choice, but it can't use on Windows directly and you need to link the readline library explicitly. You also need to install the libreadline-dev package to build and if target machine version mismatches with build machine, the program can't run. The libreadline has over 30K LOC and has two so files: libreadline.so libhistory.so, and it depends on libncurses also.

introduction1

introduction2

introduction3

introduction4

introduction5

Crossline is a small, self-contained, zero-config, MIT licensed, cross-platform, readline and libedit replacement.

When should you use Crossline: * When you need a cross-platform readline: Windows, Linux, Unix, MacOS. * When you need a simple but versatile readline: more shortcuts, advanced search, simple autocomplete, paging, cusor APIs, color APIs, embedded help, etc. * When you need a customized readline: easy to extend. * When you need a small readline to build into program.

img

Catalogue

Features and Highlights

  • Support many platforms: Windows, Linux, vt100 and xterm.
  • Support total 79 shortcuts and 40 functions.
  • Support most readline shortcuts (Emacs Standard bindings): move, edit, cut&paste, complete, history, control.
  • Support fast move between lines with Up Down and Ctrl/Alt+Up Ctrl/Alt+Down keys in multiple line edit mode.
  • Support some Windows command line shortcuts and extend some new convenient shortcuts.
  • Support history navigation, history show/clear and history save/load.
  • Support readline with initial input for editing.
  • Support autocomplete, key word help and syntax hints.
  • Support powerful interactive history search with multiple case insensitive including and excluding match patterns.
  • Support same edit shortcuts (except complete and history shortcuts) in history search mode.
  • Support color text for prompt, autocomplete, hints.
  • Support auto resizing when editing window/terminal size changed.
  • Support autocomplete, history show/search, help info paging.
  • Support cross platform paging control APIs.
  • Support cross platform cursor control APIs.
  • Support cross platform color control APIs.
  • Support convenient embedded F1 help in edit and history search mode, and you can call it anytime without losing current input.
  • Support convenient embedded Ctrl-^ keyboard debug mode to watch key code sequences.
  • Support Ctrl-C to exit edit and Ctrl-Z to suspend and resume job(Linux) in both edit and search mode.
  • Support pipe as input.
  • Pure C MIT license source code, no 3rd library dependency.
  • No any dynamic memory operations: malloc/free/realloc/new/delete/strdup/etc.
  • Very small only about 1200 LOC, and code logic is simple and easy to read.
  • Easy to customize your own shortcuts and new features.
  • Unicode is to be supported later.

Background

I'm developing a cross-platfrom command line tool which can support autocomplete and history. Readline library is the first choice, but it can't use on Windows directly and you need to link the readline library explicitly. You also need to install the libreadline-dev package to build and if target machine version mismatches with build machine, the program can't run. The libreadline has over 30K LOC and has two so files: libreadline.so libhistory.so, and it depends on libncurses also.

Then I searched and found there's a small readline replacement linenoise written by Redis's author. Linenoise is used in Redis, MongoDB, and Android. It's very small (about 1,100 LOC), but it can only run on Linux and supports few shortcuts and features. Then I found there's a more powerful library linenoise-ng used in ArangoDB. It's based on MongoDB's linenoise which enhanced original linenoise to support Windows, more shortcuts, features and Unicode. But the code is embedded in MongoDB source code, then ArangoDB ported it out, did some improvements and made it an independent library.

At first I planned to use linenoise-ng, but I found it's still big (about 4,300 LOC) and complex. It uses C++ and C together, and C/C++ dynamic memory is used also which is there in original linenoise. I think it can be much simpler, then I did some prototype verification, and use a different method to implement this brand new, simple, pure C cross-platform enhanced readline replacement library.

Shortcuts

Misc Commands

Shortcut Action
F1 Show edit shortcuts help.
Ctrl-^ Enter keyboard debugging mode.

Move Commands

Shortcut Action
Ctrl-B, Left Move back a character.
Ctrl-F, Right Move forward a character.
Up Move cursor to up line. (For multiple lines)
Ctrl-Up, Alt-Up, ESC+Up Move cursor to up line. (Ctrl-Up, Alt-Up only supports Windows/Xterm)
Down Move cursor to down line. (Ctrl-Down, Alt-Down only support Windows/Xterm)
Ctrl-Down, Alt-Down, ESC+Down Move cursor to down line. (Ctrl-Down, Alt-Down only support Windows/Xterm)
Alt-B, Ctrl-Left, Alt-Left, ESC+Left Move back a word. (Ctrl-Left, Alt-Left only support Windows/Xterm)
Alt-F, Ctrl-Right, Alt-Right, ESC+Right Move forward a word. (Ctrl-Right, Alt-Right only support Windows/Xterm)
Ctrl-A, Home Move cursor to start of line.
Ctrl-E, End Move cursor to end of line.
Ctrl-L Clear screen and redisplay line.

Edit Commands

Shortcut Action
Ctrl-H, Backspace Delete character before cursor.
Ctrl-D, DEL Delete character under cursor.
Alt-U Uppercase current or following word.
Alt-L Lowercase current or following word.
Alt-C Capitalize current or following word.
Alt-\ Delete whitespace around cursor.
Ctrl-T Transpose previous character with current character.

Cut&Paste Commands

Shortcut Action
Ctrl-K, Ctrl-End, Alt-End, ESC+End Cut from cursor to end of line. (Ctrl-End, Alt-End only support Windows/Xterm)
Ctrl-U, Ctrl-Home, Alt-Home, ESC+Home Cut from start of line to cursor. (Ctrl-Home, Alt-Home only support Windows/Xterm)
Ctrl-X Cut whole line.
Alt-Backspace, Clt-Backspace, Esc+Backspace Cut word to left of cursor. (Clt-Backspace only supports Windows/Xterm)
Alt-D, Alt-Del, Ctrl-Del, ESC+Del Cut word following cursor. (Alt-Del,Ctrl-Del only support Windows/Xterm)
Ctrl-W Cut to left till whitespace (not word).
Ctrl-Y, Ctrl-V, Insert Paste last cut text.

Complete Commands

Shortcut Action
TAB, Ctrl-I Autocomplete.
Alt-=, Alt-? List possible completions.

History Commands

Shortcut Action
Ctrl-P, Up Fetch previous line in history. (Up works when cursor in first line or end of last line)
Ctrl-N, Down Fetch next line in history. (Down works when cursor in last line)
Alt-<, PgUp Move to first line in history.
Alt->, PgDn Move to end of input history.
Ctrl-R, Ctrl-S Search history.
F4 Search history with current input.
F1 Show search help when in search mode.
F2 Show history.
F3 Clear history (need confirm).

Control Commands

Shortcut Action
Enter, Ctrl-J, Ctrl-M EOL and accept line.
Ctrl-C, Ctrl-G EOF and abort line.
Ctrl-D EOF if line is empty.
Alt-R Revert line. Undo all changes made to this line.
Ctrl-Z Suspend Job. (Linux only, fg will resume edit)

Notes

  • In multiple lines, Up/Down can move between lines. Up will fetch history when in first line or end of last line(for quick history move), and Down will fetch history when in last line.
  • For Windows and xterm, almost all shortcuts are supported.
  • For some terminal tools you need to enable Alt as meta key. SecureCRT: check Terminal->Emulation->Emacs->use ALT as meta key
  • Backspace key is 8 or 127, and Del key should use Escape code. SecurCRT: check Terminal->Emulation->Mapped Keys->Backspace sends delete
  • For vt100 and Linux terminals, many Alt-key doesn't work, and an alternate way is to press ESC first then press key, see above ESC+Key. Putty can send Alt as ESC, so no need to do this way.
  • Ctrl+S is readline shortcut to search history also, but it'll halt terminal, so don't use it with Linux system, use Ctrl+Q to exit freezing state if pressing by mistake.

Terminal Limitations

  • Some terminals only support left Alt.
  • Linux console doesn't support: Alt-?, Alt-<, Alt->.
  • SecureCRT vt100 doesn't support: Home, End, Del, Insert, PgUp, PgDn.
  • SecureCRT xterm doesn't support: Ctrl-Home, Alt-Home, Ctrl-End, Alt-End, Alt-Del, Ctrl-Del, Clt-Backspace.

Goto Top

Multiple Line Mode

Crossline supports multiple line edit mode like readline. If you input a very long line and then you want to edit some part many lines above, it's not convenient to move to the position quickly even with Ctrl+Left. I used to copy input text to a GUI editor like notepad++, then edit and copy back.

To move quickly in multiple lines, I implemented a multiple line edit mode like other editors and can use Ctrl/Alt+Up Ctrl/Alt+Down to move between lines quickly. You can also use Up Down key and to keep the original history shortcuts, the key behavior will behave as following:

  • For single line, Up Down key are history shortcuts.
  • In first line of multiple lines, Down key will move to below line, Up key will fetch previous history.
  • In middle line of multiple lines, Down key will move to below line, Up key will move to above line.
  • In last line(cursor not at end) of multiple lines, Down key will fetch next history, Up key will move to above line.
  • In last line(cursor at end) of multiple lines, Up Down key are history shortcuts. This is to keep navigating history quickly.
  • Ctrl-P Ctrl-N history shortcuts are not changed.

So if you want to use Up Down to move quickly between lines, make sure the cursor is not at end of last line, and if you want to navigate history quickly with Up Down, make sure the cursor is at end of last line in multiple lines.

Original readline supports incremental search(Ctrl-R,Ctrl-S) and none-incremental search(Alt-N,Alt-P). I tried both and think they're not convenient or efficient to use, so I implemented a brand new interactive search method.

Enter interactive history search mode

  • Ctrl-R: Search history.
  • F4: Search history with current input as search patterns.

Exit interactive history search mode

  • Ctrl-C: You can exit search mode anytime and keep your original input.
  • Ctrl-D: On empty line only.

Input patterns

  • You can use all edit shortcuts except complete and history shortcuts.
  • You can use Insert,Ctrl-Y,Ctrl-S to paste last search patterns.
  • If nothing input will show all history (F2 can show history too).

Patterns syntax

You can press F1 to get help. Patterns are separated by ' ', patter match is case insensitive. * select: choose line including select * -select: choose line excluding select * "select from": choose line including select from * -"select from": choose line excluding select from * "select from" where -"order by" -limit : choose line including select from and where and excluding order by or limit

Select history

  • If only one history is found, id 1 is provided automatically, and you just press Enter to select.
  • You can press Ctrl-C to skip choosing.
  • You can press Enter or Ctrl-D on empty line to skip choosing.

Example

SQL> <F2> // show history
-   1  hello world
-   2  select from user
-   3  from select table
-   4  SELECT from student
-   5  Select from teacher
-
-SQL> <Ctrl+R>
-Input Patterns <F1> help: select from
-   1  select from user
-   2  from select table
-   3  SELECT from student
-   4  Select from teacher
-Input history id: 3
-SQL> SELECT from student<Alt+R> // Revert line
-
-SQL> <F4>
-Input Patterns <F1> help: <Insert> // paste last history pattern: select from
-Input Patterns <F1> help: "select from"
-   1  select from user
-   2  SELECT from student
-   3  Select from teacher
-Input history id: 3
-SQL> Select from teacher<Alt+R>
-
-SQL> SELECT from<F4> // search with pattern: Select from
-Input Patterns <F1> help: SELECT from
-Input Patterns <F1> help: "SELECT from" -user -teacher
-   1  SELECT from student
-Input history id: 1
-SQL> SELECT from student
-

Goto Top

Crossline APIs

// Main API to read a line, return buf if get line, return NULL if EOF.
-char* crossline_readline (const char *prompt, char *buf, int size);
-
-// Same with crossline_readline except buf holding initial input for editing.
-char* crossline_readline2 (const char *prompt, char *buf, int size);
-
-// Set move/cut word delimiter, default is all not digital and alphabetic characters.
-void  crossline_delimiter_set (const char *delim);
-
-// Read a character from terminal without echo
-int     crossline_getch (void);
-
  • History APIs
// Save history to file
-int   crossline_history_save (const char *filename);
-
-// Load history from file
-int   crossline_history_load (const char *filename);
-
-// Show history in buffer
-void  crossline_history_show (void);
-
-// Clear history
-void  crossline_history_clear (void);
-
  • Completion APIs
typedef struct crossline_completions_t crossline_completions_t;
-typedef void (*crossline_completion_callback) (const char *buf, crossline_completions_t *pCompletions);
-
-// Register completion callback
-void  crossline_completion_register (crossline_completion_callback pCbFunc);
-
-// Add completion in callback. Word is must, help for word is optional.
-void  crossline_completion_add (crossline_completions_t *pCompletions, const char *word, const char *help);
-
-// Set syntax hints in callback
-void  crossline_hints_set (crossline_completions_t *pCompletions, const char *hints);
-

Color APIs

Color definitions

Color Note
CROSSLINE_FGCOLOR_DEFAULT If not set, it's the default value.
CROSSLINE_FGCOLOR_BLACK Set text color to black. \t is not supported in Linux terminal, same below
CROSSLINE_FGCOLOR_RED Set text color to red.
CROSSLINE_FGCOLOR_GREEN Set text color to green.
CROSSLINE_FGCOLOR_YELLOW Set text color to yellow.
CROSSLINE_FGCOLOR_BLUE Set text color to blue.
CROSSLINE_FGCOLOR_MAGENTA Set text color to magenta.
CROSSLINE_FGCOLOR_CYAN Set text color to cyan.
CROSSLINE_FGCOLOR_WHITE Set text color to white.
CROSSLINE_FGCOLOR_BRIGHT Set text color bright.
CROSSLINE_BGCOLOR_DEFAULT If not set, it's the default value.
CROSSLINE_BGCOLOR_BLACK Set background color to black. \t is not supported in Linux terminal, same below. Don't use \n in Linux terminal, same below.
CROSSLINE_BGCOLOR_RED Set background color to red.
CROSSLINE_BGCOLOR_GREEN Set background color to green.
CROSSLINE_BGCOLOR_YELLOW Set background color to yellow.
CROSSLINE_BGCOLOR_BLUE Set background color to blue.
CROSSLINE_BGCOLOR_MAGENTA Set background color to magenta.
CROSSLINE_BGCOLOR_CYAN Set background color to cyan.
CROSSLINE_BGCOLOR_WHITE Set background color to white.
CROSSLINE_BGCOLOR_BRIGHT Set background color bright.
CROSSLINE_UNDERLINE Set text to underline style.
CROSSLINE_COLOR_DEFAULT If not set, it's the default value.

CROSSLINE_FGCOLOR_DEFAULT, CROSSLINE_BGCOLOR_DEFAULT, CROSSLINE_COLOR_DEFAULT are all zero, so it's fine to leave without it or just give 0.

Color APIs

// Set text color, CROSSLINE_COLOR_DEFAULT will revert to default setting
-// `\t` is not supported in Linux terminal, same below. Don't use `\n` in Linux terminal, same below.
-void crossline_color_set (crossline_color_e color);
-
-// Set default prompt color
-void crossline_prompt_color_set (crossline_color_e color);
-
-// Add completion with color.
-void  crossline_completion_add_color (crossline_completions_t *pCompletions, const char *word, 
-                                    crossline_color_e wcolor, const char *help, crossline_color_e hcolor);
-
-// Set syntax hints with color
-void  crossline_hints_set_color (crossline_completions_t *pCompletions, const char *hints, crossline_color_e color);
-

Color Example

You can get the color example code in example2.c color_test()

img

Goto Top

Paging APIs

These APIs are used internally first, then I think they're common and can be used by CLI tools also, so make them open.

// Reset paging before starting paging control
-void crossline_paging_reset (void);
-
-// Check paging after print a line, return 1 means quit, 0 means continue
-// if you know only one line is printed, just give line_len = 1
-int  crossline_paging_check (int line_len);
-

Code in example2.c 

static void pagint_test ()
-{
-    int i;
-    crossline_paging_reset ();
-    for (i = 0; i < 256; ++i) {
-        printf ("Paging test: %3d\n", i);
-        if (crossline_paging_check (sizeof("paging test: ") + 3)) {
-            break;
-        }
-    }
-}
-

It'll work as following:

Crossline> paging
-Read line: "paging"
-Paging test:   0
-Paging test:   1
-Paging test:   2
-Paging test:   3
-Paging test:   4
-Paging test:   5
-Paging test:   6
-*** Press <Space> or <Enter> to continue . . .
-

Cursor APIs

These APIs are used internally first, then I think they're common and can be used by CLI tools also, so make them open and write some APIs which are not used by crossline at all.

// Get screen rows and columns
-void crossline_screen_get (int *pRows, int *pCols);
-
-// Clear current screen
-void crossline_screen_clear (void);
-
-// Get cursor postion (0 based)
-int  crossline_cursor_get (int *pRow, int *pCol);
-
-// Set cursor postion (0 based)
-void crossline_cursor_set (int row, int col);
-
-/* Move cursor with row and column offset, 
- *   row_off > 0 move up row_off lines, 
- *           < 0 move down abs(row_off) lines
- *           = 0 no move for row, 
- *   similar with col_off
- */
-void crossline_cursor_move (int row_off, int col_off);
-
-// Hide or show cursor
-void crossline_cursor_hide (int bHide);
-

Goto Top

Simple Example

example.c is a very simple example to use crossline.

static void completion_hook (char const *buf, crossline_completions_t *pCompletion)
-{
-    int i;
-    static const char *cmd[] = {"insert", "select", "update", "delete", "create", "drop", "show", "describe", "help", "exit", "history", "insert", "select", "update", "delete", "create", "drop", "show", "describe", "help", "exit", "history", NULL};
-
-    for (i = 0; NULL != cmd[i]; ++i) {
-        if (0 == strncmp(buf, cmd[i], strlen(buf))) {
-            crossline_completion_add (pCompletion, cmd[i], NULL);
-        }
-    }
-
-}
-
-int main ()
-{
-    char buf[256];
-
-    crossline_completion_register (completion_hook);
-    crossline_history_load ("history.txt");
-
-    while (NULL != crossline_readline ("Crossline> ", buf, sizeof(buf))) {
-        printf ("Read line: \"%s\"\n", buf);
-    }    
-
-    crossline_history_save ("history.txt");
-    return 0;
-}
-

example2.c is an enhanced example, which supports color, color test, paging test and have the edit with initial input example.

SQL Parser Example

example_sql.c, this example implements a simple SQL syntax parser with following syntax format, please read code for details.

INSERT INTO <table> SET column1=value1,column2=value2,...
-SELECT <* | column1,columnm2,...> FROM <table> [WHERE] [ORDER BY] [LIMIT] [OFFSET]
-UPDATE <table> SET column1=value1,column2=value2 [WHERE] [ORDER BY] [LIMIT] [OFFSET]
-DELETE FROM <table> [WHERE] [ORDER BY] [LIMIT] [OFFSET]
-CREATE [UNIQUE] INDEX <name> ON <table> (column1,column2,...)
-DROP {TABLE | INDEX} <name>
-SHOW {TABLES | DATABASES}
-DESCRIBE <TABLE>
-help {INSERT | SELECT | UPDATE | DELETE | CREATE | DROP | SHOW | DESCRIBE | help | exit | history}
-
You can use this example to practice the shortcuts above.

SQL> <TAB>  // show autocomplete words and help
-INSERT      Insert a record to table
-SELECT      Select records from table
-UPDATE      Update records in table
-DELETE      Delete records from table
-CREATE      Create index on table
-DROP        Drop index or table
-SHOW        Show tables or databases
-DESCRIBE    Show table schema
-help        Show help for Topic
-exit        Exit shell
-history     Show history
- *** Press <Space> or <Enter> to continue . . .
-
-SQL> help <TAB> // show autocomplete words list
-INSERT    SELECT    UPDATE    DELETE    CREATE    DROP      SHOW
-DESCRIBE  help      exit      history
-
-SQL> CREATE INDEX <TAB> // show autocomplete hints
-Please input: index name
-

img

Goto Top

Keyboard Debug

Enter keyboard debug mode

Press Ctrl-^ to enter keyboard debug mode, then you can type any key or composite key, and the code sequence will be displaced. This can be used to discover new key code sequences or debug especially for Linux system with different terminals. Note: For Windows, Alt key is a state, so it's not displayed.

Exit keyboard debug mode

Press Ctrl-C to exit.

Example

SQL> <Ctrl-^>
-Enter keyboard debug mode, <Ctrl-C> to exit debug
- 27 0x1b ( )
- 91 0x5b ([)
- 65 0x41 (A)
- 31 0x1f ( )
- 27 0x1b ( )
- 91 0x5b ([)
- 72 0x48 (H)
-

Embedded Help

Edit mode

SQL> <F1>
- Misc Commands
- +-------------------------+--------------------------------------------------+
- | F1                      |  Show edit shortcuts help.                       |
- | Ctrl-^                  |  Enter keyboard debugging mode.                  |
- +-------------------------+--------------------------------------------------+
- Move Commands
- +-------------------------+--------------------------------------------------+
- | Ctrl-B, Left            |  Move back a character.                          |
- | Ctrl-F, Right           |  Move forward a character.                       |
- | Up, ESC+Up              |  Move cursor to up line. (For multiple lines)    |
- |   Ctrl-Up, Alt-Up       |  (Ctrl-Up, Alt-Up only supports Windows/Xterm)   |
- | Down, ESC+Down          |  Move cursor to down line. (For multiple lines)  |
- |   Ctrl-Down,Alt-Down    |  (Ctrl-Down, Alt-Down only support Windows/Xterm)|
- | Alt-B, ESC+Left,        |  Move back a word.                               |
- |   Ctrl-Left, Alt-Left   |  (Ctrl-Left, Alt-Left only support Windows/Xterm)|
-*** Press <Space> or <Enter> to continue . . .
-

Search mode

SQL> <F4> or <Ctrl+R>
-Input Patterns <F1> help: <F1>
-Patterns are separated by ' ', patter match is case insensitive:
-    select:   choose line including 'select'
-    -select:  choose line excluding 'select'
-    "select from":  choose line including "select from"
-    -"select from": choose line excluding "select from"
-Example:
-    "select from" where -"order by" -limit:
-         choose line including "select from" and 'where'
-         and excluding "order by" or 'limit'
-

Extend Crossline

You can extend crossline to add new shortcuts and action easily.

Use keyboard debug mode to find the key code sequences

  • For Windows, most key has only one code, some function key has 2, first is 224 or 0, then follows another code. Alt key in Windows is a state, please refer crossline_getkey.
  • For Linux system, most function key has an escape sequences(up to 6), and one function key may have many different escape sequences for different terminal modes, so you need to test case by case.

Define new key

  • For Ctrl-key, most are single code, and you can use CTRL_KEY(key) directly.
  • For windows, most Alt-key can use ALT_KEY(num).
  • For Linux, most Alt key is escape sequences, and you can use existing macro like ESC_KEY6, example: KEY_ALT_LEFT = ESC_KEY6('1','3','D'), // xterm Esc[1;3D: Move back a word.

Support new Esc+key

Please add the conversion in crossline_key_esc2alt

Map new key to existing action

  • If it's a new different escape sequences for same key, please map the new key to main action key in crossline_key_mapping.
  • If it's a new key, please add case to crossline_readline_input.

Add new action

Please add case and action code in crossline_readline_input. You can refer existing case of similar action to write your new action. Use crossline_refreash to print line after updating buf.

Debug your code

It's not easy to debug with print or debugger tools like GDB for terminal tools, and you can use crossline_debug("xxx",...) to print debug information to file crossline_debug.txt.

Goto Top

Customized Config

Word delimiters for move and cut

Default is defined by CROSS_DFT_DELIMITER. 

#define CROSS_DFT_DELIMITER            " !\"#$%&'()*+,-./:;<=>?@[\\]^_`{|}~"
-
You can modify it or use crossline_delimiter_set to change it.

History

The history line len can be less than user buf len, and it'll cut to history line len when storing to history buf. 

#define CROSS_HISTORY_MAX_LINE       256         // Maximum history line number
-#define CROSS_HISTORY_BUF_LEN        4096        // History line length
-#define CROSS_HIS_MATCH_PAT_NUM      16          // History search pattern number
-

Completion 

#define CROSS_COMPLET_MAX_LINE        512        // Maximum completion word number
-#define CROSS_COMPLET_WORD_LEN        64         // Completion word length
-#define CROSS_COMPLET_HELP_LEN        256        // Completion word's help length
-#define CROSS_COMPLET_HINT_LEN        128        // Completion syntax hints length
-

Build and Test

On Windows, you can add the source code to a Visual Studio project to build or enter Tools Command Prompt for VS from menu to build in command line which is more efficient.

Windows MSVC

cl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example.c /Feexample.exe
-cl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example2.c /Feexample2.exe
-cl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example_sql.c /Feexample_sql.exe
-

Windows Clang

clang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example.c -o example.exe
-clang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example2.c -o example2.exe
-clang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example_sql.c -o example_sql.exe
-

GCC(Linux, MinGW, Cygwin, MSYS2)

gcc -Wall crossline.c example.c -o example
-gcc -Wall crossline.c example2.c -o example2
-gcc -Wall crossline.c example_sql.c -o example_sql
-

Linux Clang

clang -Wall crossline.c example.c -o example
-clang -Wall crossline.c example2.c -o example2
-clang -Wall crossline.c example_sql.c -o example_sql
-
  • Linenoise a small self-contained alternative to readline and libedit
  • Linenoise-ng is a fork of Linenoise that aims to add more advanced features like UTF-8 support, Windows support and other features. Uses C++ instead of C as development language.

Goto Top

\ No newline at end of file diff --git a/backup/reference/doc/crossline_color.png b/backup/reference/doc/crossline_color.png deleted file mode 100644 index c4a5f3c..0000000 Binary files a/backup/reference/doc/crossline_color.png and /dev/null differ diff --git a/backup/reference/doc/crossline_sql.png b/backup/reference/doc/crossline_sql.png deleted file mode 100644 index 33d2d28..0000000 Binary files a/backup/reference/doc/crossline_sql.png and /dev/null differ diff --git a/backup/reference/img.gif b/backup/reference/img.gif deleted file mode 100644 index 35f2a55..0000000 Binary files a/backup/reference/img.gif and /dev/null differ diff --git a/backup/reference/markdown/index.html b/backup/reference/markdown/index.html deleted file mode 100644 index b30f52a..0000000 --- a/backup/reference/markdown/index.html +++ /dev/null @@ -1,6 +0,0 @@ - Markdown Cheat Sheet - CrossDB

Markdown Cheat Sheet

Thanks for visiting The Markdown Guide!

This Markdown cheat sheet provides a quick overview of all the Markdown syntax elements. It can’t cover every edge case, so if you need more information about any of these elements, refer to the reference guides for basic syntax and extended syntax.

Basic Syntax

These are the elements outlined in John Gruber’s original design document. All Markdown applications support these elements.

Heading

H1

H2

H3

Bold

bold text

Italic

italicized text

bold italicized text

italicized text

italicized text

Line Breaks

First line with two spaces after.
And the next line.

First line with the HTML tag after.
And the next line.

Blockquote

blockquote

Blockquotes with Other Elements

The quarterly results look great!

  • Revenue was off the chart.
  • Profits were higher than ever.

Everything is going according to plan.

Ordered List

  1. First item
  2. Second item
  3. Third item
    1. Indented item
    2. Indented item
  4. Fourth item

Unordered List

  • First item
  • Second item
  • Third item
    • Indented item
    • Indented item
  • Fourth item

Mix

  1. First item
  2. Second item
  3. Third item
    • Indented item
    • Indented item

  4. Fourth item

  5. This is the first list item.

  6. Here's the second list item.

    I need to add another paragraph below the second list item.

    I need to add another paragraph below the second list item.

  7. And here's the third list item.

Code

code

Horizontal Rule

Markdown Guide

Link

Link

Markdown Guide2

can't have tab

I love supporting the EFF.
This is the Markdown Guide.
See the section on code.

URLs and Email Addresses

https://www.markdownguide.org fake@example.com

Image

img

img

Light mode

abs img

An old rock in the desert

rel img An old rock in the desert

Std img Image title

Image title

Image caption

new fig

gif

Image title Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa.

image in left
image in left
image in left
image in left
image in left
image in left
image in left
image in left

Escaping Characters

* `

Extended Syntax

These elements extend the basic syntax by adding additional features. Not all Markdown applications support these elements.

Table

Syntax Description
Header Title
Paragraph Text
Syntax Description Test Text
Header Title Here's this
Paragraph Text And more
Method Description
GET Fetch resource
PUT Update resource
DELETE Delete resource

Fenced Code Block

{
-  "firstName": "John",
-  "lastName": "Smith",
-  "age": 25
-}
-

Footnote

Here's a sentence with a footnote. 1

Here's a sentence with a footnote. 1

Heading ID

My Great Heading

Definition List

First Term
This is the definition of the first term.
Second Term
This is one definition of the second term.
This is another definition of the second term.

Strikethrough

The world is flat.

Task List

  • Write the press release
  • Update the website
  • Contact the media

Emoji

❤️ That is so funny! 😂

(See also Copying and Pasting Emoji)

Highlight

I need to highlight these very important words.

Subscript

H2O

Superscript

X2

Automatic URL Linking

http://www.example.com

http://www.example.com

Admonition

I'm Note

Some text here

Addtional Content

I'm Warning

Some text here

Addtional Content

The End

  1. This is the footnote. ↩↩

\ No newline at end of file diff --git a/backup/usecases/index.html b/backup/usecases/index.html deleted file mode 100644 index b3bfd2c..0000000 --- a/backup/usecases/index.html +++ /dev/null @@ -1 +0,0 @@ - Products - CrossDB

Products

CrossDB(CDB) is a powerful high-performance distrubuted embedded SQL RDBMS database. It's developed to improve development efficiency for embedded programmers.

CrossDB combines on-disk and in-memory data storage in a single embedded database system, so developers can optimize applications for speed and persistence, giving you flexibility to prioritize between performance, cost, power, and space-conserving storage options. Offering native and SQL APIs for C, C++, and Python.

Feature List

  • You can use CDB RDBMS to Manage Program Data efficiently.
  • You can use CDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CDB RDBMS to refactor your code conveniently.
  • You can use CDB Index to optimize performance without changing your code.
  • You can use CDB Trigger to implement Data-Driven programming paradigm.
  • You can use CDB PUBSUB to subscribe DB from other process's DB either on same host or remote host.
  • You can use CDB to implement Centralize-DB programming paradigm.
  • You can use CDB eventloop to implement event-driven programming paradigm.
  • You can use CDB RPC to build distributed service.
  • You can use CDB CLI tool to debug running program's data in off-line way.
  • You can use CDB SQL to view/create/update/delete/filter program data.
  • You can use CDB Browser to view program data.
  • You can use CDB to do DB backup restore it.
  • You can use CDB DB Change Log to view DB change history with filter, backtrace, rate-limit, expiring, etc.
  • You can use Python connector to write unit test with SQL to test program.
  • You can copy the DB folders from device and open on PC/Server with cdb-cli or Python directly.
  • Use CDB DB-Driven Mode to Build Program Logic (Table Trigger, FK, auto delete, Cascade Trigger)
  • Use CDB Python Connector to do DB-Driven Unit Test
  • Use CDB Pub/Sub to Build Distributed System (Eventloop/Timer/WorkQueue, vs IPC, OOP [priv data])
  • Use CDB Pub/Sub to Build Centralized DB-Driven System (3rd lang, DM no checkpoint)
  • CDB Serialization and RPC
  • CDB SQL Connectors (RESP3, Python)
  • CDB SQL Drivers (C, Python)
\ No newline at end of file diff --git a/blog/benchmark/crossdb-vs-lmdb/index.html b/blog/benchmark/crossdb-vs-lmdb/index.html deleted file mode 100644 index 1c0337c..0000000 --- a/blog/benchmark/crossdb-vs-lmdb/index.html +++ /dev/null @@ -1,72 +0,0 @@ - CrossDB vs LMDB Benchmark - CrossDB

CrossDB vs LMDB Benchmark

Test tool: CrossBench

DB Driver: LMDB CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core

Test Server

CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB
-HDD         : DELL PERC H755 Front SCSI Disk
-OS          : Ubuntu 20.04
-LMDB        : v0.9.24
-CrossDB     : v0.5.0
-

LMDB Config

MDB_NOSYNC | MDB_NOMETASYNC | MDB_WRITEMAP | MDB_MAPASYNC | MDB_NORDAHEAD
-

On-Disk Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s d -i 1k   -q 30m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 10k  -q 30m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 100k -q 20m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 1m   -q 10m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 10m  -q 5m  -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 100m -q 5m  -u 1m  -Q -H -c $cpu; done
-
loop="1 2 3"
-./lmdb-bench.bin -H -r 0
-for i in $loop; do ./lmdb-bench.bin -s d -i 1k   -q 10m -u 5m -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s d -i 10k  -q 10m -u 5m -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s d -i 100k -q 10m -u 5m -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s d -i 1m   -q 5m  -u 5m -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s d -i 10m  -q 2m  -u 2m -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s d -i 100m -q 1m  -u 1m -Q -H -c $cpu; done
-

Small Data Set Test

Note

LMDB DB size is fixed and preallocated, so it's faster is 1K insert test.

Large Data Set Test

On-RamDisk Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s r -i 1k   -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 10k  -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 100k -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-
loop="1 2 3"
-./lmdb-bench.bin -H -r 0
-for i in $loop; do ./lmdb-bench.bin -s r -i 1k   -q 10m -u 5m  -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s r -i 10k  -q 10m -u 5m  -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s r -i 100k -q 10m -u 5m  -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s r -i 1m   -q 5m  -u 5m  -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s r -i 10m  -q 2m  -u 2m  -Q -H -c $cpu; done
-for i in $loop; do ./lmdb-bench.bin -s r -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

Comments

\ No newline at end of file diff --git a/blog/benchmark/crossdb-vs-sqlite3/index.html b/blog/benchmark/crossdb-vs-sqlite3/index.html deleted file mode 100644 index 0ec22a3..0000000 --- a/blog/benchmark/crossdb-vs-sqlite3/index.html +++ /dev/null @@ -1,100 +0,0 @@ - CrossDB vs SQLite3 Benchmark - CrossDB

CrossDB vs SQLite3 Benchmark

Test tool: CrossBench

DB Driver: SQLite3 CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core

Note

The SQLite DB driver tries to tune SQLite performance as fast as possible and uses sqlite3_prepare_v2 to precompile SQL statements. If you have better optimization methods and it works, please let us know.

SQLite3 Config

PRAGMA synchronous = NORMAL
-PRAGMA journal_mode = WAL
-PRAGMA temp_store = memory
-PRAGMA optimize
-
PRAGMA synchronous = OFF
-PRAGMA journal_mode = WAL
-PRAGMA temp_store = memory
-PRAGMA optimize
-
PRAGMA synchronous = OFF
-PRAGMA journal_mode = OFF
-PRAGMA temp_store = memory
-PRAGMA optimize
-

Test Server

CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB
-HDD         : DELL PERC H755 Front SCSI Disk
-OS          : Ubuntu 20.04
-SQLite3     : v3.31.1
-CrossDB     : v0.5.0
-

On-Disk Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s d -i 1k   -q 30m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 10k  -q 30m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 100k -q 20m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 1m   -q 10m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 10m  -q 5m  -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s d -i 100m -q 5m  -u 1m  -Q -H -c $cpu; done
-
loop="1 2 3"
-./sqlite-bench.bin -H -r 0
-for i in $loop; do ./sqlite-bench.bin -s d -i 1k   -q 1k  -u 1k  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s d -i 10k  -q 1m  -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s d -i 100k -q 1m  -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s d -i 1m   -q 1m  -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s d -i 10m  -q 1m  -u 1m  -Q -H -c $cpu; done
-# This test is very very very slow
-for i in $loop; do ./sqlite-bench.bin -s d -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

On-RamDisk Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s r -i 1k   -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 10k  -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 100k -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s r -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-
loop="1 2 3"
-./sqlite-bench.bin -H -r 0
-for i in $loop; do ./sqlite-bench.bin -s r -i 1k   -q 1m -u 500k  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s r -i 10k  -q 1m -u 500k  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s r -i 100k -q 1m -u 300k  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s r -i 1m   -q 500k -u 250k -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s r -i 10m  -q 500k -u 250k -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s r -i 100m -q 500k -u 250k -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

In-Memory Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-
loop="1 2 3"
-./sqlite-bench.bin -H -r 0
-for i in $loop; do ./sqlite-bench.bin -s m -i 1k   -q 2m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s m -i 10k  -q 2m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s m -i 100k -q 2m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s m -i 1m   -q 1m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s m -i 10m  -q 1m -u 1m  -Q -H -c $cpu; done
-for i in $loop; do ./sqlite-bench.bin -s m -i 100m -q 1m -u 1m  -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

Comments

\ No newline at end of file diff --git a/blog/benchmark/crossdb-vs-stlhmap/index.html b/blog/benchmark/crossdb-vs-stlhmap/index.html deleted file mode 100644 index c08fdc6..0000000 --- a/blog/benchmark/crossdb-vs-stlhmap/index.html +++ /dev/null @@ -1,55 +0,0 @@ - CrossDB vs STL Hashmap Benchmark - CrossDB

CrossDB vs STL Hashmap Benchmark

Test tool: CrossBench

DB Driver: STLHashmap CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core, To make test fair, STL map uses read-write lock as CrossDB is thread-safe with read-write lock also.

Note

Official STL Hashmap is unordered_map, which is the fastest data container in CPP. This test is to show how close CrossDB performance to STL Hashmap and CrossDB query can be faster than Hashmap in large data set test.

Test Server

CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB
-HDD         : DELL PERC H755 Front SCSI Disk
-OS          : Ubuntu 20.04
-STL         : GCC v10.2.1
-CrossDB     : v0.5.0
-

In-Memory Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-
loop="1 2 3"
-./stlhmap-bench.bin -H -r 0
-for i in $loop; do ./stlhmap-bench.bin -s m -i 1k   -q 60m -u 60m -Q -H -c $cpu; done
-for i in $loop; do ./stlhmap-bench.bin -s m -i 10k  -q 60m -u 60m -Q -H -c $cpu; done
-for i in $loop; do ./stlhmap-bench.bin -s m -i 100k -q 30m -u 40m -Q -H -c $cpu; done
-for i in $loop; do ./stlhmap-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./stlhmap-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./stlhmap-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

Comments

\ No newline at end of file diff --git a/blog/benchmark/crossdb-vs-stlmap/index.html b/blog/benchmark/crossdb-vs-stlmap/index.html deleted file mode 100644 index 3f9235a..0000000 --- a/blog/benchmark/crossdb-vs-stlmap/index.html +++ /dev/null @@ -1,55 +0,0 @@ - CrossDB vs STL Map Benchmark - CrossDB

CrossDB vs STL Map Benchmark

Test tool: CrossBench

DB Driver: STLMap CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core, To make test fair, STL map uses read-write lock as CrossDB is thread-safe with read-write lock also.

Test Server

CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB
-HDD         : DELL PERC H755 Front SCSI Disk
-OS          : Ubuntu 20.04
-STL         : GCC v10.2.1
-CrossDB     : v0.5.0
-

In-Memory Database Benchmark

Test Script

loop="1 2 3"
-./crossdb-bench.bin -H -r 0
-for i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done
-
loop="1 2 3"
-./stlmap-bench.bin -H -r 0
-for i in $loop; do ./stlmap-bench.bin -s m -i 1k   -q 40m -u 40m -Q -H -c $cpu; done
-for i in $loop; do ./stlmap-bench.bin -s m -i 10k  -q 20m -u 20m -Q -H -c $cpu; done
-for i in $loop; do ./stlmap-bench.bin -s m -i 100k -q 10m -u 10m -Q -H -c $cpu; done
-for i in $loop; do ./stlmap-bench.bin -s m -i 1m   -q 5m  -u 5m  -Q -H -c $cpu; done
-for i in $loop; do ./stlmap-bench.bin -s m -i 10m  -q 2m  -u 2m  -Q -H -c $cpu; done
-for i in $loop; do ./stlmap-bench.bin -s m -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done
-

Small Data Set Test

Large Data Set Test

Comments

\ No newline at end of file diff --git a/blog/why-new-embdb/index.html b/blog/why-new-embdb/index.html deleted file mode 100644 index bff62ba..0000000 --- a/blog/why-new-embdb/index.html +++ /dev/null @@ -1,32 +0,0 @@ - Why Need a New Embedded Database - CrossDB

Why Need a New Embedded Database

TBD

Comments

\ No newline at end of file diff --git a/client/api-c/index.html b/client/api-c/index.html new file mode 100755 index 0000000..638378b --- /dev/null +++ b/client/api-c/index.html @@ -0,0 +1,189 @@ + C APIs - CrossDB

C APIs

APIs List

API Descritpion
xdb_conn_t* xdb_open (const char *path) Open a connection
void xdb_close (xdb_conn_t *pConn) Close a connection
xdb_res_t* xdb_exec (xdb_conn_t* pConn, const char *sql) Execute SQL statement
xdb_res_t* xdb_pexec (xdb_conn_t* pConn, const char *sql, ...) Execute formatted SQL statement
xdb_res_t* xdb_next_result (xdb_conn_t *pConn) Get next SQL statement result
bool xdb_more_result (xdb_conn_t* pRes) Check is there more result
void xdb_free_result (xdb_res_t* pRes) Free result set
xdb_col_t* xdb_column_meta (uint64_t meta, uint16_t iCol) Fetch one row
xdb_row_t* xdb_fetch_row (xdb_res_t* pRes) Fetch one row
int xdb_column_int (uint64_t meta, void *pRow, uint16_t iCol) Get int column from row
float xdb_column_float (uint64_t meta, void *pRow, uint16_t iCol) Get float/double column from row
const char * xdb_column_str (uint64_t meta, void *pRow, uint16_t iCol) Get string column from row
xdb_stmt_t* xdb_stmt_prepare (xdb_conn_t* pConn, const char *sql) Prepare statement
xdb_ret xdb_bind_int (xdb_stmt_t *pStmt, uint16_t para_id, int val) Bind int value
xdb_ret xdb_bind_float (xdb_stmt_t *pStmt, uint16_t para_id, float val) Bind float value
xdb_ret xdb_bind_str (xdb_stmt_t *pStmt, uint16_t para_id, const char *str) Bind string value
xdb_res_t* xdb_stmt_exec (xdb_stmt_t *pStmt) Execute prepared SQL statement
void xdb_stmt_close (xdb_stmt_t *pStmt) Free prepared SQL statement
xdb_ret xdb_begin (xdb_conn_t* pConn) Begin transaction
xdb_ret xdb_commit (xdb_conn_t* pConn) Commit transaction
xdb_ret xdb_rollback (xdb_conn_t* pConn) Rollback transaction
int xdb_print_row (uint64_t meta, void *pRow, int format) Print row to console
const char * xdb_errmsg (xdb_res_t *pRes) Get error/information message
const char * xdb_version () Get CrossDB version string

xdb_open

Open a connection and create/open a Database.

xdb_conn_t*
+xdb_open (const char *path);
+
+// TBD
+xdb_conn_t*
+xdb_open2 (const char *path, uint32_t flags);
+
  • If path is :memory: will create or open a memory database, else create or open a on-disk database. This database will be the default DB of this connection.
  • If path is NULL: Only open a connection. User has to use OPEN DATABASE '[path/]db_name' or CREATE DATABASE '[path/]db_name' or USE DATABASE db_name.
  • One thread can use only one connection. One connection can only be used by one thread
  • All opened databases are shared with all opened connections, and they can USE DATABASE db_name to switch connection default DB.

xdb_close

Create a connection.

void
+xdb_close (xdb_conn_t *pConn);
+

xdb_exec

Execute SQL statement and return result set.

xdb_res_t*
+xdb_exec (xdb_conn_t* pConn, const char *sql);
+
+xdb_res_t*
+xdb_exec2 (xdb_conn_t *pConn, const char *sql, int len);
+
  • A valid xdb_res_t pointer is returned always.
  • row count: pRes->row_count
  • column count: pRes->col_count
  • column meta: pRes->row_meta, use xdb_col_t* xdb_column_meta (uint64_t meta, uint16_t iCol) to get column meta.

xdb_pexec

Execute formatted SQL statement and return result set.

xdb_res_t*
+xdb_pexec (xdb_conn_t *pConn, const char *sql, ...);
+
  • A valid xdb_res_t pointer is returned always.

Note

If sql contains %, should use escape \%.

xdb_next_result

Get next SQL statement result 

xdb_res_t*
+xdb_next_result (xdb_conn_t *pConn);
+

xdb_more_result

Check is there more result 

bool
+xdb_more_result (xdb_conn_t *pConn);
+

xdb_free_result

Free result set 

void
+xdb_free_result (xdb_res_t *pRes);
+

Note

You only need to call xdb_free_result for statements more than 0 rows.

xdb_column_meta

Get column meta.

const xdb_col_t* 
+xdb_column_meta (uint64_t meta, uint16_t iCol)
+

Note

This is an inline function.

xdb_fetch_row

Fetch one row and return row list array.

xdb_row_t*
+xdb_fetch_row (xdb_res_t *pRes);
+

xdb_column_int

Get int column from row. 

int 
+xdb_column_int (uint64_t meta, void *pRow, uint16_t iCol);
+
+int64_t 
+xdb_column_int64 (uint64_t meta, void *pRow, uint16_t iCol);
+

Note

If you know the detailed format, you can access the pointer directly. *(int8_t*)pVal[iCol], *(int16_t*)pVal[iCol], *(int32_t*)pVal[iCol], *(int64_t*)pVal[iCol]

xdb_column_float

Get float column from row.

float
+xdb_column_float (uint64_t meta, xdb_row_t *pRow, uint16_t iCol);
+
+double 
+xdb_column_double (uint64_t meta, void *pRow, uint16_t iCol);
+

Note

If you know the detailed format, you can access the pointer directly. *(float*)pVal[iCol], *(double*)pVal[iCol]

xdb_column_str

Get string column from row.

const char*
+xdb_column_str (uint64_t meta, xdb_row_t *pRow, uint16_t iCol);
+
+const char*
+xdb_column_str2 (uint64_t meta, xdb_row_t *pRow, uint16_t iCol, int *pLen);
+

Note

You can access the pointer directly: *(const char*)pVal[iCol], get the length *(uint16_t*)(pVal[iCol]-2)

xdb_stmt_prepare

xdb_stmt_t* 
+xdb_stmt_prepare (xdb_conn_t* pConn, const char *sql);
+

xdb_bind_int

Binds an int value to the prepared statement at the specified parameter index.

xdb_ret
+xdb_bind_int (xdb_stmt_t *pStmt, uint16_t para_id, int val);
+
+xdb_ret
+xdb_bind_int64 (xdb_stmt_t *pStmt, uint16_t para_id, int64_t val);
+

xdb_bind_float

Binds a double value to the prepared statement at the specified parameter index.

xdb_ret
+xdb_bind_float (xdb_stmt_t *pStmt, uint16_t para_id, float val);
+
+xdb_ret
+xdb_bind_double (xdb_stmt_t *pStmt, uint16_t para_id, double val)
+

xdb_bind_str

Binds a string value to the prepared statement at the specified parameter index.

xdb_ret
+xdb_bind_str (xdb_stmt_t *pStmt, uint16_t id, const char *str, int len);
+
+xdb_ret
+xdb_bind_str2 (xdb_stmt_t *pStmt, uint16_t para_id, const char *str, int len)
+

xdb_stmt_exec

Execute a prepared statement.

xdb_res_t*
+xdb_stmt_exec (xdb_stmt_t *pStmt);
+

xdb_stmt_close

Close a prepared statement.

void
+xdb_stmt_close (xdb_stmt_t *pStmt);
+

xdb_begin

Begin a transaction.

xdb_ret
+xdb_begin (xdb_conn_t* pConn);
+
  • For embedded local connection, always return XDB_OK.

xdb_commit

Commit a transaction.

xdb_ret
+xdb_commit (xdb_conn_t* pConn);
+
  • For embedded local connection, always return XDB_OK.

xdb_rollback

Rollback a transaction.

xdb_ret
+xdb_rollback (xdb_conn_t* pConn);
+
  • For embedded local connection, always return XDB_OK.

xdb_print_row

Print row to console.

int 
+xdb_print_row (uint64_t meta, xdb_row_t *pRow, int format);
+

xdb_errmsg

Get error/information message in result.

const char *
+xdb_errmsg (xdb_res_t *pRes);
+

xdb_version

Get CrossDB version string.

const char*
+xdb_version();
+

Types

xdb_errno_e

Error Code 

typedef enum {
+    XDB_OK,
+    XDB_ERROR,
+    XDB_E_PARAM,
+    XDB_E_STMT,
+    XDB_E_NODB,
+    XDB_E_NOTFOUND,
+    XDB_E_EXISTS,
+    XDB_E_FULL,
+    XDB_E_CONSTRAINT,
+    XDB_E_AUTH,
+    XDB_E_MEMORY,
+    XDB_E_FILE,
+    XDB_E_SOCK,
+} xdb_errno_e;
+

xdb_type_t

Data Types 

typedef enum {
+    XDB_TYPE_NULL       = 0,  // 1 bit
+    XDB_TYPE_TINYINT    = 1,  // 1 byte
+    XDB_TYPE_SMALLINT   = 2,  // 2 bytes
+    XDB_TYPE_INT        = 3,  // 4 bytes
+    XDB_TYPE_BIGINT     = 4,  // 8 bytes
+    XDB_TYPE_UTINYINT   = 5,  // 1 byte
+    XDB_TYPE_USMALLINT  = 6,  // 2 bytes
+    XDB_TYPE_UINT       = 7,  // 4 bytes
+    XDB_TYPE_UBIGINT    = 8,  // 8 bytes
+    XDB_TYPE_FLOAT      = 9,  // 4 bytes
+    XDB_TYPE_DOUBLE     = 10, // 8 bytes
+    XDB_TYPE_TIMESTAMP  = 11, // 8 bytes
+    XDB_TYPE_CHAR       = 12, // fixed-length string(at most 65535 byte)
+    XDB_TYPE_BINARY     = 13, // fixed-length binary(at most 65535 byte)
+    XDB_TYPE_VCHAR      = 14, // varied-length string(at most 65535 byte)
+    XDB_TYPE_VBINARY    = 15, // varied-length binary(at most 65535 byte)
+    XDB_TYPE_MAX        = 21
+} xdb_type_t;
+

Structures

xdb_res_t

Result Set 

typedef struct xdb_res_t {
+    uint32_t    len_type;       // MSB 4bit are type xdb_restype_t
+    uint16_t    errcode;        // 4
+    uint16_t    status;         // 6 xdb_status_t
+
+    uint32_t    meta_len;       // 8
+    uint16_t    col_count;      // 12
+    uint8_t     stmt_type;      // 14 SQL type(create/delete/drop/show/select/delete/update...)
+    uint8_t     rsvd;
+
+    uint64_t    row_count;      // 2*8 SELECT/SHOW
+    uint64_t    affected_rows;  // 3*8 INSERT/UPDATE/DELETE
+    uint64_t    insert_id;      // 4*8 INSERT
+    uint64_t    col_meta;       // 5*8 xdb_meta_t, <ptr:ptr off: 0 following is meta>
+    uint64_t    row_data;       // 6*8 xdb_rowlist_t, ptr: base ptr or error str or infomation xdb_msg_t
+    uint64_t    data_len;       // 7*8
+} xdb_res_t;
+

xdb_msg_t

Return Message 

typedef struct {
+    uint32_t    len_type;       // LSB 4bit are type
+    uint16_t    len;
+    char        msg[];
+} xdb_msg_t;
+

xdb_meta_t

Query Meta information 

typedef struct {
+    uint32_t    len_type;       // LSB 4bit are type
+    uint16_t    col_count;      // 3*4
+    uint16_t    null_off;       // 3*4+2
+    uint16_t    row_size;
+    uint16_t    rsvd;
+    uint64_t    col_list;       // xdb_col_t list
+    xdb_col_t   cols[];
+} xdb_meta_t;
+

xdb_col_t

Query Column information 

typedef struct {
+    uint16_t    col_len;    // colum total len
+    uint8_t     col_type;   // xdb_type_t
+    uint8_t     col_rsvd;
+    uint32_t    col_off;
+    uint16_t    col_rsvd2;
+    uint8_t     col_nmlen;
+    char        col_name[];
+} xdb_col_t;
+

xdb_rowdat_t

Query row data information 

typedef struct {
+    uint32_t    len_type;       // LSB 4bit are type
+    uint8_t     rowdat[];
+} xdb_rowdat_t;
+

xdb_rowlist_t

typedef uint64_t xdb_row_t; // xdb_rowdat_t
+
+typedef struct {
+    uint32_t    rl_count;
+    uint32_t    rl_curid;
+    xdb_row_t   rl_pRows[];
+} xdb_rowlist_t;
+

Comments

\ No newline at end of file diff --git a/client/api-python/index.html b/client/api-python/index.html new file mode 100755 index 0000000..801892e --- /dev/null +++ b/client/api-python/index.html @@ -0,0 +1,31 @@ + Python - CrossDB

Python

TBD

Comments

\ No newline at end of file diff --git a/community/index.html b/community/index.html new file mode 100755 index 0000000..6659edd --- /dev/null +++ b/community/index.html @@ -0,0 +1,31 @@ + Community - CrossDB

Community

Report Bugs

Github Issues

Require Fetures

Github Fetures

Discussion

Github Discussion

Contribution

In order to keep CrossDB quality and high-performance, this project does not accept patches. If you would like to suggest a change and you include a patch as a proof-of-concept, that would be great. However, please do not be offended if we rewrite your patch from scratch.

Following contributions are welcome:

  • Language bindings: Python, Java, Go, CSharp, Javascript, PHP, etc
  • Test and report bugs

Comments

\ No newline at end of file diff --git a/develop/connect/index.html b/develop/connect/index.html new file mode 100755 index 0000000..738fa64 --- /dev/null +++ b/develop/connect/index.html @@ -0,0 +1,75 @@ + Connect - CrossDB

Connect to CrossDB

  • One thread can use only one connection. One connection can only be used by one thread.
  • All opened databases are shared with all opened connections, and they can USE DATABASE db_name to switch connection's default DB.

C APIs

  • Open a connection
xdb_conn_t *pConn = xdb_open (NULL);
+

Note

User has to use OPEN DATABASE '[path/]db_name' or CREATE DATABASE '[path/]db_name' or USE DATABASE db_name.

  • Open a connection and create a default memory database if not exist
xdb_conn_t *pConn = xdb_open (":memory:");
+
  • Open a connection and create a on-disk database school if not exist in current folder as the default database
xdb_conn_t *pConn = xdb_open ("school");
+

Comments

\ No newline at end of file diff --git a/develop/ddl/index.html b/develop/ddl/index.html new file mode 100755 index 0000000..d1de9f4 --- /dev/null +++ b/develop/ddl/index.html @@ -0,0 +1,55 @@ + Data Model - CrossDB

Data Model

Create and Use a Database

-- Create database in current folder
+CREATE DATABASE school;
+-- Create database in specified folder (must exist)
+CREATE DATABASE '/var/dbpath/school';
+-- Create in-memory database 
+CREATE DATABASE test ENGINE=memory;
+
// Create database in current folder
+xdb_res_t *pRes = xdb_exec (pConn, "CREATE DATABASE school");
+// Create database in specified folder (must exist)
+xdb_res_t *pRes = xdb_exec (pConn, "CREATE DATABASE '/var/dbpath/school'");
+// Create in-memory database 
+xdb_res_t *pRes = xdb_exec (pConn, "CREATE DATABASE test ENGINE=memory");
+

Create a Table

CREATE TABLE IF NOT EXISTS student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(256), INDEX (name));
+CREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(256), INDEX (name));
+CREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name));
+
xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE TABLE IF NOT EXISTS student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(256), INDEX (name))");
+xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(256), INDEX (name))");
+xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name))");
+

Create Index

CREATE INDEX idx_name ON student (age);
+CREATE INDEX idx_name ON teacher (age);
+CREATE INDEX idx_name ON student (author);
+
xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE INDEX idx_name ON student (age)");
+xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE INDEX idx_name ON teacher (age)");
+xdb_res_t *pRes = pRes = xdb_exec (pConn, "CREATE INDEX idx_name ON student (author)");
+

Comments

\ No newline at end of file diff --git a/develop/dml/index.html b/develop/dml/index.html new file mode 100755 index 0000000..260dde9 --- /dev/null +++ b/develop/dml/index.html @@ -0,0 +1,35 @@ + Data Manipulation - CrossDB

Data Manipulation

Insert Rows

INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95);
+INSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')", 'He is a boy.\nHe likes playing football.\nWe all like him!');
+
pRes = xdb_exec (pConn, "INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)");
+pRes = xdb_pexec (pConn, "INSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')", "He is a boy.\nHe likes playing football.\nWe all like him!");
+

Update Rows

TBD

Delete Rows

TBD

Comments

\ No newline at end of file diff --git a/develop/multi-stmts/index.html b/develop/multi-stmts/index.html new file mode 100755 index 0000000..01bc866 --- /dev/null +++ b/develop/multi-stmts/index.html @@ -0,0 +1,31 @@ + Multi-Statements - CrossDB

Multi-Statements

TBD

Comments

\ No newline at end of file diff --git a/develop/prep-stmts/index.html b/develop/prep-stmts/index.html new file mode 100755 index 0000000..68c3fae --- /dev/null +++ b/develop/prep-stmts/index.html @@ -0,0 +1,31 @@ + Prepared Statements - CrossDB

Prepared Statements

TBD

Comments

\ No newline at end of file diff --git a/develop/query/index.html b/develop/query/index.html new file mode 100755 index 0000000..c4da1ce --- /dev/null +++ b/develop/query/index.html @@ -0,0 +1,45 @@ + Query Rows - CrossDB

Query Rows

TBD

Comments

\ No newline at end of file diff --git a/develop/transaction/index.html b/develop/transaction/index.html new file mode 100755 index 0000000..6d1cec7 --- /dev/null +++ b/develop/transaction/index.html @@ -0,0 +1,31 @@ + Transaction - CrossDB

Transaction

Reader-Writeer MVCC

Current CrossDB uses table-level readwrite lock, only one writer connection thread can modify the table, other writer connection threads will be blocked. However the reader connection threads won't be blocked and they'll see the old values. When the writer connection change is committed, the new value is visible by these reader connections (read committed).
In short writer before commit will not block readers to read old values

Auto-commit

TBD

Begin

TBD

Commit

TBD

Rollback

TBD

Comments

\ No newline at end of file diff --git a/docs/api/api/index.html b/docs/api/api/index.html deleted file mode 100644 index b1ef38b..0000000 --- a/docs/api/api/index.html +++ /dev/null @@ -1,31 +0,0 @@ - API List - CrossDB

API List

Database Definition APIs

API Descritpion
cross_ret cross_dbCreate (cross_db_h *phDb, const char *dbName, uint32_t flags) Create or Open Database
cross_ret cross_dbClose (cross_db_h hDb, uint32_t flags) Close Database
cross_ret cross_dbDrop (cross_db_h hDb, uint32_t flags) Drop Database
cross_ret cross_dbTblCreate (cross_db_h hDb, cross_tbl_h *phTbl, const char *tblName, const cross_field_t *pFields, const char *priKey, uint32_t flags) Create or Open Table
cross_ret cross_dbTblDrop (cross_tbl_h hTbl, uint32_t flags) Drop Table
cross_ret cross_dbIdxCreate (cross_tbl_h hTbl, const char *idxName, const char *fldsStr, uint32_t flags) Create Index on Table
cross_ret cross_dbIdxDrop (cross_tbl_h hTbl, const char *idxName, uint32_t flags) Drop Table Index

Database Manipulation APIs

API Descritpion
cross_ret cross_dbInsertRow (cross_tbl_h hTbl, void *pRow, uint32_t flags) Intert row into Table
cross_ret cross_dbReplaceRow (cross_tbl_h hTbl, void *pInRow, uint32_t flags) Insert/update row
cross_ret cross_dbGetRowByPK (cross_tbl_h hTbl, const void *pInRow, void *pOutRow, uint32_t flags) Get one row by Primary Key
cross_ret cross_dbUpdRowByPK (cross_tbl_h hTbl, const void *pInRow, const void *pUpdFlds, void *pUpdRow, uint32_t flags) Update row by Primary Key
cross_ret cross_dbDelRowByPK (cross_tbl_h hTbl, void *pInRow, uint32_t flags) Delete row by Primary Key
cross_ret cross_dbGetOneRow (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, void *pOutRow, uint32_t flags) Get one row by match fields
cross_rowid cross_dbUpdateRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, const void *pUpdFlds, const void *pUpdRow, uint32_t flags) Update rows by match fields
cross_rowid cross_dbDeleteRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags) Delete rows by match fields
cross_rowid cross_dbGetRowsCount (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags) Get rows count by match fields
cross_rowid cross_dbQueryRows (cross_tbl_h hTbl, cross_cursor_h *phCursor, const void *pMatFlds, const void *pMatRow, uint32_t flags) Query rows into cursor
cross_ret cross_cursorGetNextRow (cross_cursor_h hCursor, void *pOutRow, uint32_t flags) Get next row in cursor
cross_rowid cross_cursorGetCount (cross_cursor_h hCursor, uint32_t flags) Get cursor rows count
cross_ret cross_cursorClose (cross_cursor_h hCursor, uint32_t flags) Close cursor
cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags) Begin Transaction
cross_ret cross_dbTransCommit (cross_db_h hDb, uint32_t flags) Commit Transaction
cross_ret cross_dbTransRollback (cross_db_h hDb, uint32_t flags) Rollback Transaction

Misc APIs

API Descritpion
const char* cross_errMsg (cross_db_h hDb, uint32_t flags) Return error code message
cross_ret cross_fieldsCreate (cross_tbl_h hTbl, cross_fields_h *phFlds, const char *FldsStr, uint32_t flags) Create fields handle
void cross_fieldsFree (cross_fields_h hFlds) Free fields handle
cross_ret cross_matchCreate (cross_tbl_h hTbl, cross_match_h *phMatch, const char *MatchStr, uint32_t flags) Create match handle
void cross_matchFree (cross_match_h hMatch) Free match handle

Comments

\ No newline at end of file diff --git a/docs/api/ddl/index.html b/docs/api/ddl/index.html deleted file mode 100644 index ef2888a..0000000 --- a/docs/api/ddl/index.html +++ /dev/null @@ -1,47 +0,0 @@ - Database Definition APIs - CrossDB

Database Definition APIs

Database

cross_dbCreate

Create or Open Database 

cross_ret 
-cross_dbCreate (cross_db_h *phDb, const char *dbName, uint32_t flags);
-
Parameters

Arg Type Descritpion
phDb out DB Handle
dbName in DB Name(can with Path)
flags [in] See following tables
Flag Descritpion
CROSS_ONDISK DB is on persistent disk, survive with power cycle (Default)
CROSS_RAMDISK DB is on ramdisk/tmpfs/ramfs, survive with process restart, lose after power cycle
CROSS_INMEM DB is in memory, survie when process is runnig, lose after process terminates
CROSS_EXCLUSIVE DB is used exclusively by single process (Default)
CROSS_SHARED DB is shared by multiple processes
CROSS_AUTOLOCK DB will do lock automatically (Default)
CROSS_OPEN Open DB (don't create if not exist)

Returns

  • CROSS_OK: The database was created/opened successfully
  • Other: Decode with cross_errMsg

cross_dbClose

Close Database 

cross_ret 
-cross_dbClose (cross_db_h hDb, uint32_t flags);
-
Parameters

Arg Type Descritpion
hDb in DB Handle
flags [in] Not used

Returns

  • CROSS_OK: The database was closed successfully
  • Other: Decode with cross_errMsg

Warning

DB handle can't be used after close.

cross_dbDrop

Drop Database 

cross_ret 
-cross_dbDrop (cross_db_h hDb, uint32_t flags);
-
Parameters

Arg Type Descritpion
hDb in DB Handle
flags [in] Not used

Returns

  • CROSS_OK: The database was dropped successfully
  • Other: Decode with cross_errMsg

Warning

All tables and indexes will be removed. DB handle can't be used after drop.

Table

cross_dbTblCreate

Create or Open Table 

cross_ret 
-cross_dbTblCreate (cross_db_h hDb, cross_tbl_h *phTbl, const char *tblName,
-    const cross_field_t *pFields, const char *priKey, uint32_t flags);
-

Parameters

Arg Type Descritpion
hDb in DB Handle
phDb out Table Handle
tblName in Table Name
pFields [in] Table Schema Fields
priKey [in] Primary Key cloumn list seperated by ,
flags [in] See following tables
Flag Descritpion
CROSS_RBTREE Primary Key is RBTREE
CROSS_OPEN Open DB (don't create if not exist)

Returns

  • CROSS_OK: The table was created/opened successfully
  • Other: Decode with cross_errMsg

cross_dbTblDrop

Drop Table 

cross_ret 
-cross_dbTblDrop (cross_tbl_h hTbl, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
flags [in] Not used

Returns

  • CROSS_OK: The table was dropped successfully
  • Other: Decode with cross_errMsg

Warning

Table handle can't be used after drop.

Index

cross_dbIdxCreate

Create Index on Table 

cross_ret 
-cross_dbIdxCreate (cross_tbl_h hTbl, const char *idxName, 
-                    const char *fldsStr, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
idxName in index name(table scope)
fldsStr in cloumn list seperated by ,
flags [in] See following tables
Flag Descritpion
CROSS_HASH index type HASH (default)
CROSS_RBTREE index type RBTREE
CROSS_UNIQUE index is Unique

Returns

  • CROSS_OK: The index was created successfully
  • Other: Decode with cross_errMsg

cross_dbIdxDrop

Drop Table Index 

cross_ret 
-cross_dbIdxDrop (cross_tbl_h hTbl, const char *idxName, uint32_t flags);
-
Parameters

Arg Type Descritpion
hTbl in Table Handle
idxName in index name(table scope)
flags [in] Not used

Returns

  • CROSS_OK: The index was dropped successfully
  • Other: Decode with cross_errMsg

Comments

\ No newline at end of file diff --git a/docs/api/dml/index.html b/docs/api/dml/index.html deleted file mode 100644 index 701efe7..0000000 --- a/docs/api/dml/index.html +++ /dev/null @@ -1,68 +0,0 @@ - Database Manipulation APIs - CrossDB

Database Manipulation APIs

Single Row Manipulation

cross_dbInsertRow

Intert row to Table 

cross_ret 
-cross_dbInsertRow (cross_tbl_h hTbl, void *pRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pRow in Row buffer to insert
pValFlds [in] Not used
flags [in] Not used

Returns

  • CROSS_OK: The row was inserted successfully
  • Other: Decode with cross_errMsg

Description

  • If table has Primary Key or unique index, it must be valid and unique.

cross_dbReplaceRow

Insert row to Table if not exist else replace row 

cross_ret 
-cross_dbReplaceRow (cross_tbl_h hTbl, void *pInRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pRow in Row buffer to insert/update
pNullFlds in Not used
flags [in] Not used

Returns

  • CROSS_OK: The row was inserted/updated successfully
  • Other: Decode with cross_errMsg

Description

  • If table has Primary Key or unique index, it must be valid and unique.

cross_dbGetRowByPK

Get one row by Primary Key 

cross_ret 
-cross_dbGetRowByPK (cross_tbl_h hTbl, const void *pInRow, void *pOutRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pInRow in Row buffer with Primary Key
pOutRow [out] Row buffer to return DB row
flags [in] Not used

Returns

  • CROSS_OK: Get row successfully
  • CROSS_E_NOTFOUND: The row is not found
  • Other: Decode with cross_errMsg

Description

  • pOutRow = NULL can be used to check if row exists by Primary Key
  • pOutRow can = pInRow

cross_dbUpdRowByPK

Update row by Primary Key 

cross_ret 
-cross_dbUpdRowByPK (cross_tbl_h hTbl, const void *pInRow, 
-                    const void *pUpdFlds, void *pUpdRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pInRow in Row buffer with Primary Key
pUpdFlds [in] Fields to update
pUpdRow in Row buffer to upate
flags [in] Not used

Returns

  • CROSS_OK: Update row successfully
  • CROSS_E_NOTFOUND: The row is not found
  • Other: Decode with cross_errMsg

Description

  • pUpdFlds can be cloumn list seperated by ,
  • pUpdFlds can be handle created by cross_fieldsCreate
  • pUpdFlds = NULL will update whole DB row
  • pUpdRow can = pInRow

cross_dbDelRowByPK

Delete row by Primary Key 

cross_ret 
-cross_dbDelRowByPK (cross_tbl_h hTbl, void *pInRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pInRow in Row buffer with Primary Key
flags [in] Not used

Returns

  • CROSS_OK: Delete row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg

cross_dbGetOneRow

Get one row by match fields 

cross_ret 
-cross_dbGetOneRow (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, 
-                    void *pOutRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pMatFlds in Fields to match
pMatRow in Row buffer with match row
pOutRow [out] Row buffer to return DB row
flags [in] Not used

Returns

  • CROSS_OK: Get row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg

Description

  • pMatFlds can be cloumn list seperated by ,, they are AND together.
    • fld: equal to match dbRow.fld = pMatRow.fld.
    • fld!=: unequal match dbRow.fld != pMatRow.fld.
    • fld>: greater than match dbRow.fld > pMatRow.fld.
    • fld>=: greater than or equal to match dbRow.fld >= pMatRow.fld.
    • fld<: less match dbRow.fld < pMatRow.fld.
    • fld<=: leas than or equal to match dbRow.fld < pMatRow.fld.
  • pMatFlds can be handle created by cross_matchCreate
  • pOutRow = NULL can be used to check if row exists

Multiple Rows Manipulation

cross_dbUpdateRows

Update rows by match fields 

cross_rowid 
-cross_dbUpdateRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, 
-                    const void *pUpdFlds, const void *pUpdRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pMatFlds in Fields to match
pMatRow in Row buffer with match row
pUpdFlds [in] Upate specified fields
pUpdRow in Row buffer to update
flags [in] Not used

Returns

  • >=0: Rows count updated successfully
  • Other: Decode with cross_errMsg

Description

  • pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow
  • pMatFlds can be handle created by cross_matchCreate
  • pUpdFlds can be cloumn list seperated by ,,
  • pUpdFlds can be handle created by cross_fieldsCreate
  • pUpdFlds = NULL updates whole row

cross_dbDeleteRows

Delete rows by match fields 

cross_rowid 
-cross_dbDeleteRows (cross_tbl_h hTbl, const void *pMatFlds, 
-                    const void *pMatRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pMatFlds in Fields to match
pMatRow in Row buffer with match row
flags [in] Not used

Returns

  • >=0: Rows count deleted successfully
  • Other: Decode with cross_errMsg

Description - pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow - pMatFlds can be handle created by cross_matchCreate

cross_dbGetRowsCount

Get rows count by match fields 

cross_rowid 
-cross_dbGetRowsCount (cross_tbl_h hTbl, const void *pMatFlds, 
-                        const void *pMatRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
pMatFlds in Fields to match
pMatRow in Row buffer with match row
flags [in] Not used

Returns

Description - pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow - pMatFlds can be handle created by cross_matchCreate

Cursor

cross_dbQueryRows

Query rows into cursor 

cross_rowid 
-cross_dbQueryRows (cross_tbl_h hTbl, cross_cursor_h *phCursor, const void *pMatFlds, 
-                    const void *pMatRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
phCursor io|out Return Cursor handle or
pMatFlds in Fields to match
pMatRow in Row buffer with match row
flags [in] CROSS_REUSE means phCursor points to a valid cursor handle and it'll be reused

Returns

Description

cross_cursorGetNextRow

Get next row in cursor 

cross_ret 
-cross_cursorGetNextRow (cross_cursor_h hCursor, void *pOutRow, uint32_t flags);
-

Parameters

Arg Type Descritpion
hCursor in Cursor Handle
pOutRow [out] Row buffer to return DB row
flags [in] Not used

Returns

  • CROSS_OK: Get next row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg

cross_cursorGetCount

Get cursor rows count 

cross_rowid 
-cross_cursorGetCount (cross_cursor_h hCursor, uint32_t flags);
-

Parameters

Arg Type Descritpion
hCursor in Cursor Handle
flags [in] Not used

Returns

cross_cursorClose

Close cursor 

cross_ret 
-cross_cursorClose (cross_cursor_h hCursor, uint32_t flags);
-

Parameters

Arg Type Descritpion
hCursor in Cursor Handle
flags [in] Not used

Returns

Warning

Cursor handle can't be used after close.

Transaction

cross_dbTransBegin

Begin Transaction 

cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags);
-

Parameters

Arg Type Descritpion
hDb in DB Handle
flags [in] Not used

Returns

  • CROSS_OK: Begin transaction successfully
  • Other: Decode with cross_errMsg

Note

If there's onging transaction, it'll be commited first.

cross_dbTransCommit

Commit Transaction 

cross_ret 
-cross_dbTransCommit (cross_db_h hDb, uint32_t flags);
-

Parameters

Arg Type Descritpion
hDb in DB Handle
flags [in] Not used

Returns

  • CROSS_OK: Commit transaction successfully
  • Other: Decode with cross_errMsg

cross_dbTransRollback

Rollback Transaction 

cross_ret 
-cross_dbTransRollback (cross_db_h hDb, uint32_t flags);
-

Parameters

Arg Type Descritpion
hDb in DB Handle
flags [in] Not used

Returns

  • CROSS_OK: Rollback transaction successfully
  • Other: Decode with cross_errMsg

Comments

\ No newline at end of file diff --git a/docs/api/misc/index.html b/docs/api/misc/index.html deleted file mode 100644 index e328764..0000000 --- a/docs/api/misc/index.html +++ /dev/null @@ -1,39 +0,0 @@ - Misc APIs - CrossDB

Misc APIs

cross_errMsg

Return error code message 

const char*
-cross_errMsg (cross_ret ret);
-

cross_fieldsCreate

Create fields handle 

cross_ret   
-cross_fieldsCreate (cross_tbl_h hTbl, cross_fields_h *phFlds, const char *FldsStr, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
phFlds out Fields Handle
FldsStr in Cloumn list seperated by ,
flags [in] Not used

Returns

  • CROSS_OK: Create handle successfully
  • Other: Decode with cross_errMsg

cross_fieldsFree

Free fields handle 

void        cross_fieldsFree (cross_fields_h hFlds);
-

cross_matchCreate

Create match handle 

cross_ret   
-cross_matchCreate (cross_tbl_h hTbl, cross_match_h *phMatch, const char *MatchStr, uint32_t flags);
-

Parameters

Arg Type Descritpion
hTbl in Table Handle
phMatch out Match Handle
MatchStr in Match list seperated by ,
flags [in] Not used

Returns

  • CROSS_OK: Create handle successfully
  • Other: Decode with cross_errMsg

Description

  • MatchStr cloumn list seperated by ,, they are AND together. Refer dml.md#cross_dbGetOneRow
    • fld: equal to match dbRow.fld = pMatRow.fld.
    • fld!=: unequal match dbRow.fld != pMatRow.fld.
    • fld>: greater than match dbRow.fld > pMatRow.fld.
    • fld>=: greater than or equal to match dbRow.fld >= pMatRow.fld.
    • fld<: less match dbRow.fld < pMatRow.fld.
    • fld<=: leas than or equal to match dbRow.fld < pMatRow.fld.

cross_matchFree

Free match handle 

void        cross_matchFree (cross_match_h hMatch);
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/database/index.html b/docs/crossdb/database/index.html deleted file mode 100644 index e204233..0000000 --- a/docs/crossdb/database/index.html +++ /dev/null @@ -1,72 +0,0 @@ - Database - CrossDB

Database

CrosDB Database is a collection of tables.

Steps to define a database

  • Decide where to store the database: on disk, on ramdisk or in memory.
  • Decide the database is accessed by single process or multiple proceses.
  • Use cross_dbCreate to create or open database.
  • Use Schema to define table scheam.
  • Decide Table Primary Key: fields list, index type.
  • Use cross_dbTblCreate to create or open table with schema defined above and provide the primary key.
  • Decide how many secondary indexes: fields list, index type, whether unique.
  • Use cross_dbIdxCreate to create table secondary index.

Storage Mode

On-Disk Database

  • Database is on persistent disk, survive with power cycle.
  • If you have data persistency requirements, please use this type. The performance is ver high.
  • You need use transaction to guarantee data integrity even power cycle happened.

RamDisk Database

  • Database is on ramdisk tmpfs ramfs(CROSS_RAMDISK), survive with process restart, lose after power cycle.
  • This is designed for process runtime databse and the transaction performance is higher than On-Disk database.
  • The performance is almost the same with In-Memory database.
  • For Linux embedded system, this is the prefered database as the database can be viewed and stayed there even process crashed.

In-Memory Database

  • Database is in memory(CROSS_INMEM), survie when process is runnig, lose after process terminates.
  • This is not recommented, but if you don't want the process runtime database visible, it's the solution.

Access Mode

Exclusive

  • Database is accessed exclusively by single process with best performance.

Shared

  • Databse is accessd by multiple process(CROSS_SHARED), then performance will be low as file lock is used to access DB which can recover lock when process crashed.

Note

Database is accessed exclusively by default.

Guide

  • Database is stored on disk by default.
  • If path(absoulte or relative) is not provided, DB is stored in current folder.

Example

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
#define CHECK(ret,str)  \
-    if (ret < 0) {  printf (str": %s\n", cross_errMsg(ret)); return -1; }
-
-cross_ret       ret;
-cross_db_h      hDb;
-
-// Create on disk database in current folder for single process access
-ret = cross_dbCreate (&hDb, "ondiskdb", 0);
-CHECK (ret, "Failed to create database");
-
-// Create on disk database for multiple process access
-ret = cross_dbCreate (&hDb, "cdb_data/shareddb", CROSS_SHARED);
-CHECK (ret, "Failed to create database");
-
-// Create ramdisk database for single process access
-ret = cross_dbCreate (&hDb, "/tmp/cdb_data/mydb", CROSS_RAMDISK);
-CHECK (ret, "Failed to create database");
-
-// Create in memory database
-ret = cross_dbCreate (&hDb, "imdb", CROSS_INMEM);
-CHECK (ret, "Failed to create database");
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/dml/index.html b/docs/crossdb/dml/index.html deleted file mode 100644 index 5a6bc63..0000000 --- a/docs/crossdb/dml/index.html +++ /dev/null @@ -1,633 +0,0 @@ - Data Manipulation Guide - CrossDB

Data Manipulation Guide

Define Database

Handle and Macros

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
cross_ret       ret;
-route_t         route;  
-cross_rowid     count;
-cross_db_h      hDb = NULL;
-cross_tbl_h     hRtTbl = NULL;
-cross_fields_h  hFlagsFlds = NULL;
-cross_match_h   hNexthopMat = NULL, hNexthopNeMat = NULL;
-
-#define CHECK(ret,str)      if (ret < 0) {  printf (str": %s\n", cross_errMsg(ret)); return -1; }
-#define EXPECT(val,exp,str) if (val != exp) {   printf ("%s: %d != %d\n", str, val, exp); return -1; } \
-                            else  { printf ("%s: %d\n", str, val); }
-
-#define IP4ADDR(a,b,c,d)        (uint32_t)((a)<<24|(b)<<16|(c)<<8|(d))
-#define IP4STR(ip)              ip>>24,(ip>>16)&0xff,(ip>>8)&0xff,ip&0xff
-#define DUMP_ROUTE(str, route)  printf (str"%d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n",     \
-                        IP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);
-

Define Schema

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
typedef struct {
-uint32_t            prefix;
-uint8_t             mask;
-uint32_t            nexthop;
-uint8_t             metric;
-char                intf[16];
-time_t              birth;
-uint32_t            flags;
-} route_t;
-
-#undef  CROSS_STRUCT_NAME
-#define CROSS_STRUCT_NAME   route_t
-cross_field_t   route_schema[] = {
-CROSS_FIELD (prefix,    UINT,   IPv4, 0),
-CROSS_FIELD (mask,      UINT,   DFT,  0),
-CROSS_FIELD (nexthop,   UINT,   IPv4, 0),
-CROSS_FIELD (metric,    UINT,   DFT,  0),
-CROSS_FIELD (intf,      CHAR,   DFT,  0),
-CROSS_FIELD (birth,     UINT,   TS,   0),
-CROSS_FIELD (flags,     UINT,   HEX,  0),
-CROSS_END (route_t)
-};
-

Create DB

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
printf ("\n============ Create DB & Table ============\n");
-
-printf ("  Create database : tutorial\n");
-
-ret = cross_dbCreate (&hDb, "db_data/tutorial", 0);
-CHECK (ret, "Failed to create DB: tutorial");
-
-printf ("  Create table: route (PrimaryKey: prefix,mask)\n");
-ret = cross_dbTblCreate (hDb, &hRtTbl, "route", route_schema, "prefix,mask", 0);
-CHECK (ret, "Failed to create table: route");
-
-printf ("  Create index on nexthop: idx_nexthop\n");
-ret = cross_dbIdxCreate (hRtTbl, "idx_nexthop", "nexthop", 0);
-CHECK (ret, "Failed to create index: idx_nexthop");
-
-// Delete all rows
-cross_dbDeleteRows (hRtTbl, NULL, NULL, 0);
-
-ret = cross_fieldsCreate (hRtTbl, &hFlagsFlds, "flags", 0);
-CHECK (ret, "Failed to create fields: flags");
-ret = cross_matchCreate (hRtTbl, &hNexthopMat, "nexthop", 0);
-CHECK (ret, "Failed to create fields: nexthop");
-ret = cross_matchCreate (hRtTbl, &hNexthopNeMat, "nexthop!=", 0);
-CHECK (ret, "Failed to create fields: nexthop!=");
-

Manipulation Database

Insert Rows

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
printf ("\n============ Insert Rows ============\n");
-
-printf ("  Insert route 192.168.1.0/24->192.168.1.254\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(192,168,1,254);
-route.metric    = 1;
-route.flags     = 0;
-strcpy (route.intf, "eth1");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert route 192.168.1.0/24");
-
-printf ("  Insert route 10.1.1.0/24->10.1.2.254\n");
-route.prefix    = IP4ADDR(10,1,1,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.metric    = 2;
-route.flags     = 0;
-strcpy (route.intf, "eth2");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert route 10.1.1.0/24");
-
-printf ("  Use Replace to insert route 10.1.2.0/24->10.1.2.254\n");
-route.prefix    = IP4ADDR(10,1,2,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.metric    = 1;
-route.flags     = 0;
-strcpy (route.intf, "eth2");
-route.birth     = time (NULL);
-ret = cross_dbReplaceRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to replace route 10.1.2.0/24");
-

Query Rows

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
-44
printf ("\n============ Query Rows ============\n");
-
-// Get all rows count
-count = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);
-EXPECT (count, 3, "  Total routes");
-
-// Get rows count where nexthop=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {
-    count = cross_dbGetRowsCount (hRtTbl, "nexthop", &route, 0);
-} else {
-    count = cross_dbGetRowsCount (hRtTbl, hNexthopMat, &route, 0);
-}
-EXPECT (count, 2, "  Nexthop=10.1.2.254 routes");
-
-// Get single route 192.168.1.0/24 by PrimaryKey
-memset (&route, 0, sizeof(route));
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0); 
-CHECK (ret, "Failed to get route 192.168.1.0/24 by PrimaryKey");
-DUMP_ROUTE ("  Get single route: ", route);
-
-// Get one row where nexthop=10.1.2.254
-memset (&route, 0, sizeof(route));  
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {
-    ret = cross_dbGetOneRow (hRtTbl, "nexthop", &route, &route, 0);
-} else {
-    ret = cross_dbGetOneRow (hRtTbl, hNexthopMat, &route, &route, 0);
-}
-CHECK (ret, "Failed to get one route where nexthop=10.1.2.254");
-DUMP_ROUTE ("  Get one route where nexthop=10.1.2.254: ", route);
-
-// Get one row where nexthop!=10.1.2.254
-memset (&route, 0, sizeof(route));  
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {
-    ret = cross_dbGetOneRow (hRtTbl, "nexthop!=", &route, &route, 0);
-} else {
-    ret = cross_dbGetOneRow (hRtTbl, hNexthopNeMat, &route, &route, 0);
-}
-CHECK (ret, "Failed to get one route where nexthop!=10.1.2.254");
-DUMP_ROUTE ("  Get one route where nexthop!=10.1.2.254: ", route);
-

Update Rows

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
-44
-45
-46
-47
-48
-49
-50
-51
-52
-53
-54
-55
-56
-57
-58
-59
-60
-61
-62
-63
-64
printf ("\n============ Update Rows ============\n");
-
-printf ("  Update single route 192.168.1.0/24 by Primary Key: set flags ->1\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 1;
-if (!optimize) {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, "flags", &route, 0); 
-} else {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); 
-}
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-// verify
-route.flags     = 0;
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-EXPECT (route.flags, 1, "    Update flags");
-DUMP_ROUTE ("    Get single route: ", route);
-
-printf ("  Update single route 192.168.1.0/24 by Primary Key: update whole row\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-CHECK (ret, "Failed to get route 192.168.1.0/24 by PrimaryKey");
-route.flags     = 2;
-route.metric    = 2;
-ret = cross_dbUpdRowByPK (hRtTbl, &route, NULL, &route, 0); 
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-// verify
-route.flags     = 0;
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-EXPECT (route.flags, 2, "    Update flags");
-EXPECT (route.metric, 2, "    Update metric");
-DUMP_ROUTE ("    Get single route: ", route);
-
-printf ("  Use Replace to Update single route 192.168.1.0/24 by Primary Key: set flags ->3\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 3;
-ret = cross_dbReplaceRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to replace route 192.168.1.0/24");
-// verify
-route.flags     = 0;
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-EXPECT (route.flags, 3, "    Update flags");
-DUMP_ROUTE ("    Get single route: ", route);
-
-// Update routes where nexthop=10.1.2.254: set flags 0->4
-route.nexthop   = IP4ADDR(10,1,2,254);
-route.flags     = 4;
-if (!optimize) {
-    count = cross_dbUpdateRows (hRtTbl, "nexthop", &route, "flags", &route, 0);
-} else {
-    count = cross_dbUpdateRows (hRtTbl, hNexthopMat, &route, hFlagsFlds, &route, 0);
-}
-EXPECT (count, 2, "  Update nexthop=10.1.2.254 set flags=4 rows");
-
-// Update All routes set flags 0->5
-route.flags     = 5;
-if (!optimize) {
-    count = cross_dbUpdateRows (hRtTbl, NULL, NULL, "flags", &route, 0);
-} else {
-    count = cross_dbUpdateRows (hRtTbl, NULL, NULL, hFlagsFlds, &route, 0);
-}
-EXPECT (count, 3, "  Update all routes set flags=4 rows");
-

Use Cursor

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
printf ("\n============ Cursor Query ============\n");
-
-// Use cursor to get all routes
-cross_cursor_h hCursor;
-count = cross_dbQueryRows (hRtTbl, &hCursor, NULL, &route, 0);
-EXPECT (count, 3, "  Query All routes");
-while (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {
-    DUMP_ROUTE ("    route: ", route);
-}
-
-// Reuse cursor to get routes where nexthop=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {
-    count = cross_dbQueryRows (hRtTbl, &hCursor, "nexthop", &route, CROSS_REUSE);
-} else {
-    count = cross_dbQueryRows (hRtTbl, &hCursor, hNexthopMat, &route, CROSS_REUSE);
-}   
-EXPECT (count, 2, "  Query nexthop=10.1.2.254 routes");
-while (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {
-    DUMP_ROUTE ("    route: ", route);
-}
-
-// Reuse cursor to get routes where nexthop!=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {    
-    count = cross_dbQueryRows (hRtTbl, &hCursor, "nexthop!=", &route, CROSS_REUSE);
-} else {
-    count = cross_dbQueryRows (hRtTbl, &hCursor, hNexthopNeMat, &route, CROSS_REUSE);
-}
-EXPECT (count, 1, "  Query nexthop!=10.1.2.254 routes");
-while (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {
-    DUMP_ROUTE ("    route: ", route);
-}
-
-cross_cursorClose (hCursor, 0);
-

Use Transaction

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
-23
-24
-25
-26
-27
-28
-29
-30
-31
-32
-33
-34
-35
-36
-37
-38
-39
-40
-41
-42
-43
printf ("\n============ Transaction ============\n");
-
-printf ("  Commit\n");
-ret = cross_dbTransBegin (hDb, 0);
-CHECK (ret, "Failed to begin transaction");
-printf ("    Update single route 192.168.1.0/24 by Primary Key: set flags to 8\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 8;
-if (!optimize) {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, "flags", &route, 0); 
-} else {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); 
-}
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-ret = cross_dbTransCommit (hDb, 0);
-CHECK (ret, "Failed to commit transaction");
-// verify
-route.flags     = 0;
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-EXPECT (route.flags, 8, "    Update flags");
-DUMP_ROUTE ("    Get single route: ", route);
-
-printf ("  Rollback\n");
-ret = cross_dbTransBegin (hDb, 0);
-CHECK (ret, "Failed to begin transaction");
-printf ("    Update single route 192.168.1.0/24 by Primary Key: set flags 0->9\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 9;
-if (!optimize) {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, "flags", &route, 0); 
-} else {
-    ret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); 
-}
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-ret = cross_dbTransRollback (hDb, 0);
-CHECK (ret, "Failed to commit transaction");
-// verify
-route.flags     = 0;
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);
-EXPECT (route.flags, 8, "    Rollback to orignal flags");
-DUMP_ROUTE ("    Get single route: ", route);
-

Delete Rows

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
-21
-22
printf ("\n============ Delete Rows ============\n");
-
-printf ("  Delete single route 192.168.1.0/24 by Primary Key\n");
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbDelRowByPK (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to delete route 192.168.1.0/24 by Primary Key");
-// verify
-count = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);
-EXPECT (count, 2, "    Total routes");
-
-// Delete routes where nexthop=10.1.2.254
-route.nexthop   = IP4ADDR(10,1,2,254);
-if (!optimize) {
-    count = cross_dbDeleteRows (hRtTbl, "nexthop", &route, 0);
-} else {
-    count = cross_dbDeleteRows (hRtTbl, hNexthopMat, &route, 0);
-}
-EXPECT (count, 2, "  Delete nexthop=10.1.2.254 rows");
-// verify
-count = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);
-EXPECT (count, 0, "    Total routes");
-

Close DB

1
-2
printf ("\n============ Close DB ============\n");
-cross_dbClose (hDb, 0);
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/indexes/index.html b/docs/crossdb/indexes/index.html deleted file mode 100644 index 7c48d76..0000000 --- a/docs/crossdb/indexes/index.html +++ /dev/null @@ -1,66 +0,0 @@ - Index - CrossDB

Index

CrossDB Index is used to implement database Primary Key and accelerate query.

Index Type

HASH Index

  • HASH index has highest O(1) performance. It's optimized a lot to achieve best performance.
  • HASH index can dynamically scale with table rows number to get high performance.
  • HASH index only supports exact match.

RBTREE Index

  • Use flag CROSS_RBTREE
  • RBTREE index is almost the same with popular RDBMS BTree, which is core index engine of RDBMS.
  • RBTREE index can support exact match, range match and leftmost prefix match with multiple-column.

Guide

  • Default index is HASH type.
  • If one query matches both HASH index and RBTREE index, then Hash index is selected in most cases.
  • You can create HASH index for high performance query and RBTREE index for normal performance queries.
  • Index is not free, it occupies space and all inexes may be updated during INSERT/UPDATE/DELET row.
  • If index is unique, please set CROSS_UNIQUE.
  • For unique index, either insert replace update will check row is unique.
  • You can do any query on any fields combination without index.
  • Create index only when the query calls frequency is high and requires performance.

Example

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
#define CHECK(ret,str)  \
-    if (ret < 0) {  printf (str": %s\n", cross_errMsg(ret)); return -1; }
-
-// Create hash index
-ret = cross_dbIdxCreate (hRtTbl, "idx_nexthop", "nexthop", 0);
-CHECK (ret, "Failed to create index");
-
-// Create rbtree index
-ret = cross_dbIdxCreate (hRtTbl, "idx_nexthop", "nexthop", 0);
-CHECK (ret, "Failed to create index");
-
-// Create unique hash index
-ret = cross_dbIdxCreate (hRtTbl, "idx_route", "vrf,ipAddr", CROSS_UNIQUE);
-CHECK (ret, "Failed to create index");
-
-// Create unique rbtree index
-ret = cross_dbIdxCreate (hRtTbl, "idx_route", "vrf,ipAddr", CROSS_UNIQUE|CROSS_RBTREE);
-CHECK (ret, "Failed to create index");
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/schema/index.html b/docs/crossdb/schema/index.html deleted file mode 100644 index 40f1606..0000000 --- a/docs/crossdb/schema/index.html +++ /dev/null @@ -1,155 +0,0 @@ - Schema - CrossDB

Schema

Schema Definition

Schem is used to define CrossDB table and it's defined based on C struct with some CrossDB macro.

Basic Types

CrossDB Type and C Type Map

CrossDB Type Description C Type
INT integer char, short, int, long long, int8_t, int16_t, int32_t, int64_t
UINT unsinged integer unsigned char, unsigned short, unsigned int, unsigned long long, uint8_t uint16_t, uint32_t, uint64_t
FLOAT floating-point float, double
CHAR charater string char []
BYTE byte array unsiged char [], uint8_t [], struct

CrossDB Format

CrossDB Type CrossDB Formt Description
INT/UINT DFT default decimal
FLOAT DFT default floating-point
CHAR DFT default charater string
BYTE DFT default hexadecimal string
INT/UINT BOOL boolean
UINT HEX hexadecimal
INT/UINT TIMESTAMP/TS Timestamp: time_t, uin32_t, uint64_t
BTYE MAC MAC addrss: uint8_t [6], struct ether_addr
UINT IPv4 Host endian IPv4 address: uint32_t
BYTE IPv4 Network endian IPv4 address: struct in_addr, uint32_t
BYTE IPv6 Network endian IPv4 address: uint8_t [16], struct in_addr6

Example

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
typedef struct {
-    char            val_char;
-    int64_t         val_int64;
-    uint16_t        val_u16;
-    uint8_t         val_u8;
-    float           val_float;
-    double          val_double;
-    char            val_str[16];
-    uint8_t         val_byte[64];
-
-    char            val_bool;
-    uint16_t        flags;
-    time_t          birth;
-    unsigned char   mac[6];
-    uint32_t        ipAddrHost;
-    struct in_addr  ipAddrNet;
-    uint8_t         ip6Addr[16];
-} basic_types_t;
-

Set CROSS_STRUCT_NAME to the struct name basic_types_t for CROSS_FIELD to define the schema entry. CROSS_END is used to mark the end of schema definition.

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
#undef  CROSS_STRUCT_NAME
-#define CROSS_STRUCT_NAME   basic_types_t
-cross_field_t   basic_types_schema[] = {
-    CROSS_FIELD (val_char,      INT,    DFT,  0),
-    CROSS_FIELD (val_int64,     INT,    DFT,  0),
-    CROSS_FIELD (val_u16,       UINT,   DFT,  0),
-    CROSS_FIELD (val_u8,        UINT,   DFT,  0),
-    CROSS_FIELD (val_float,     FLOAT,  DFT,  0),
-    CROSS_FIELD (val_double,    FLOAT,  DFT,  0),
-    CROSS_FIELD (val_str,       CHAR,   DFT,  0),
-    CROSS_FIELD (val_byte,      BYTE,   DFT,  0),
-    CROSS_FIELD (val_bool,      UINT,   BOOL, 0),
-    CROSS_FIELD (flags,         UINT,   HEX,  0),
-    CROSS_FIELD (birth,         UINT,   TS,   0),
-    CROSS_FIELD (mac,           BYTE,   MAC,  0),
-    CROSS_FIELD (ipAddrHost,    UINT,   IPv4, 0),
-    CROSS_FIELD (ipAddrNet,     BYTE,   IPv4, 0),
-    CROSS_FIELD (ip6Addr,       BYTEE,  IPv6, 0),
-    CROSS_END (basic_types_t)
-};
-

To keep struct and schema in header file, you can define them together like following way.

Define macro ROUTE_SCHEMA() in header file 

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
-20
// In header file
-typedef struct {
-    uint32_t            prefix;
-    uint8_t             mask;
-    uint32_t            nexthop;
-    uint8_t             metric;
-    char                intf[16];
-    uint32_t            birth;
-    uint32_t            flags;
-} route_t;
-
-#dfine ROUTE_SCHEMA()   \   
-    CROSS_FIELD (prefix,    UINT,   IPv4, 0), \
-    CROSS_FIELD (mask,      UINT,   DFT,  0), \
-    CROSS_FIELD (nexthop,   UINT,   IPv4, 0), \
-    CROSS_FIELD (metric,    UINT,   DFT,  0), \
-    CROSS_FIELD (intf,      CHAR,   DFT,  0), \
-    CROSS_FIELD (birth,     UINT,   TS,   0), \
-    CROSS_FIELD (flags,     UINT,   HEX,  0), \
-    CROSS_END (route_t)
-

Define schema in source file. CROSS_END can be define in route_schema also then you can reuse the macro ROUTE_SCHEMA() or combine them. 

1
-2
-3
-4
-5
-6
    // In source file
-    #undef  CROSS_STRUCT_NAME
-    #define CROSS_STRUCT_NAME   route_t
-    cross_field_t   route_schema[] = {
-        ROUTE_SCHEMA()
-    }
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/table/index.html b/docs/crossdb/table/index.html deleted file mode 100644 index 3d318c8..0000000 --- a/docs/crossdb/table/index.html +++ /dev/null @@ -1,52 +0,0 @@ - Table - CrossDB

Table

CrosDB Table is a collection of rows(record or tuple) and columns.

  • Table schema is mapped from C Struct, so this is a kind of C Struct ORM(Object Relational Mapping).
  • The C struct is expanded into each single fields for composite data type like array, nested struct etc.
  • Each table can have only one Primary key.
  • Each table can hve many secondray indexes.

Primary Key(PK)

  • Each table can have only one Primray Key, which is used to identify the row uniquely.
  • Primay Key column list shouldn't change during the row lifecycle.
  • Primary Key is implemented with unique HASH index for high performance. You can change to RBTREE as well.

Guide

  • If table exists and pFields is different with table schema, table will be upgraded automatically.
  • If you want to know if table exists, you can use CROSS_OPEN to get handle first.
  • Primary Key is HASH type by default to achieve highest performance.
  • If you don't care about performance, you can set CROSS_RBTREE to create RBTREE type Primary Key.
  • If you need both exact match and range match for Primary Key, you can create another RBTREE index with same column list.

Example

 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
// Create table with PrimaryKey="prefix,mask", HASH Type
-ret = cross_dbTblCreate (hDb, &hRtTbl, "route", route_schema, "prefix,mask", 0);
-CHECK (ret, "Failed to create route table");
-
-// Create table with PrimaryKey="prefix,mask", RBTREE Type
-ret = cross_dbTblCreate (hDb, &hRtTbl, "route", route_schema, "prefix,mask", CROSS_RBTREE);
-CHECK (ret, "Failed to create route table");
-
-// Get table handle
-ret = cross_dbTblCreate (hDb, &hRtTbl, "route", NULL, NULL, CROSS_OPEN);
-CHECK (ret, "Failed to get route table");
-

Comments

\ No newline at end of file diff --git a/docs/crossdb/transaction/index.html b/docs/crossdb/transaction/index.html deleted file mode 100644 index d915bb6..0000000 --- a/docs/crossdb/transaction/index.html +++ /dev/null @@ -1,31 +0,0 @@ - Transaction - CrossDB

Transaction

ACID

In the context of transaction processing, the acronym ACID refers to the four key properties of a transaction: atomicity, consistency, isolation, and durability.

Atomicity All changes to data are performed as if they are a single operation. That is, all the changes are performed, or none of them are. For example, in an application that transfers funds from one account to another, the atomicity property ensures that, if a debit is made successfully from one account, the corresponding credit is made to the other account.

Consistency Data is in a consistent state when a transaction starts and when it ends. For example, in an application that transfers funds from one account to another, the consistency property ensures that the total value of funds in both the accounts is the same at the start and end of each transaction.

Isolation The intermediate state of a transaction is invisible to other transactions. As a result, transactions that run concurrently appear to be serialized. For example, in an application that transfers funds from one account to another, the isolation property ensures that another transaction sees the transferred funds in one account or the other, but not in both, nor in neither.

Durability After a transaction successfully completes, changes to data persist and are not undone, even in the event of a system failure. For example, in an application that transfers funds from one account to another, the durability property ensures that the changes made to each account will not be reversed.

API

API Descritpion
cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags) Begin Transaction
cross_ret cross_dbTransCommit (cross_db_h hDb, uint32_t flags) Commit Transaction
cross_ret cross_dbTransRollback (cross_db_h hDb, uint32_t flags) Rollback Transaction

AutoCommit

  • For on-disk database, autocommit is enabled for single row and multiple rows DMLs by default.
  • For ramdisk and inmem database, autocommit is enabled for single row DMLs only by default. For multiple rows DMLs, if process crashed, some rows may be updated/deleted, some may not, so if you need strict ACID, you have to call transaction APIs manually. This is to improve performance and let user to control the behavior by need.

Comments

\ No newline at end of file diff --git a/docs/get-started/index.html b/docs/get-started/index.html deleted file mode 100644 index e25f3f1..0000000 --- a/docs/get-started/index.html +++ /dev/null @@ -1,81 +0,0 @@ - Get Started - CrossDB

Get Started

Download

Choose the OS and CPU, then Download here.

Linux/FreeBSD package file list

File Descritpion
crossdb.h The only header file
libcrossdb.so The only shared library
crossdb-cli CrossDB command line tool
examples/ CrossDB example code

Windows package file list

File Descritpion
crossdb.h The only header file
crossdb.dll The only shared library
crossdb.lib For build with MSCV compiler
crossdb-cli.exe CrossDB command line tool
examples/ CrossDB example code

MacOS package file list

File Descritpion
crossdb.h The only header file
libcrossdb.dylib The only shared library for X64 and AMD64
crossdb-cli CrossDB command line tool
examples/ CrossDB example code

Linux/FreeBSD

Run Examples

cd examples
-
-./build.sh example.c
-./example.bin
-
-./build.sh tutorial.c
-./tutorial.bin
-

Build in your project

  • You can check examples/build.sh to build the library in your project folder.

  • Yo can also install CrossDB globally and use it as common library.

    1. Install libcrossdb.so to /usr/lib

    #include "crossdb.h"
-// your code
-

    2. Build this way: gcc my.c -lcrossdb -pthread -ldl

Windows

Run Examples with MSVC

Start Visual Studio command line from menu x64 Native Tools Command Prompt for VS 20xx

Enter CrossDB package folder

cd examples
-
-build example.c
-example.exe
-
-build tutorial.c
-tutorial.exe
-

Run Examples with MINGW64

cd examples
-
-./build.sh example.c
-./example.bin
-
-./build.sh tutorial.c
-./tutorial.bin
-

Note

You can run in git bash or MSYS2 MINGW64 If you only have MINGW64, you an run in commandline

gcc -o example.exe -Wall -O2 example.c -I.. crossdb.dll

Build in your project

  • For MINGW64, you can just use crossdb.h and crossdb.dll to compile.
  • For Visual Studio, You can add crossdb.h crossdb.lib and crossdb.dll to your project.

MacOS

Run Examples

cd examples
-
-./build.sh example.c
-./example.bin
-
-./build.sh tutorial.c
-./tutorial.bin
-

Build in your project

  • You can check examples/build.sh to build the library in your project folder.

  • Yo can also install CrossDB globally and use it as common library.

    1. Install libcrossdb.dylib to /usr/local/lib

    #include "crossdb.h"
-// your code
-

    2. Build this way: clang my.c -lcrossdb

Use CrossDB CLI

../crossdb-cli db_data
-
-CrossDB> insert into route set prefix=1.1.1.1 mask=24 nexthop=1.1.1.254 metric=5 intf='eth1'
-
-CrossDB> select * from route
-ID  prefix   mask  nexthop    metric  intf  birth                flags
-==  =======  ====  =========  ======  ====  ===================  =====
-1   1.1.1.1  24    1.1.1.254  5       eth1  1970-01-01T08:00:00  0x0
-
-Qualified Rows: 1       Select Rows: 1    Use time 2us QPS 500000
-

More commands can be found here

Comments

\ No newline at end of file diff --git a/docs/introduction/index.html b/docs/introduction/index.html deleted file mode 100644 index eaa93d2..0000000 --- a/docs/introduction/index.html +++ /dev/null @@ -1,51 +0,0 @@ - Introduction - CrossDB

Introduction

CrossDB is a powerful high-performance embedded RDBMS database. It's developed to improve development efficiency for embedded application or embedded system development.

Features

  • Support Multiple OS Platform: Linux/Windows/MacOS/FreeBSD etc
  • Support X86/ARM/PPC/MIPS etc CPU
  • Support OnDisk/In-memory/RamDisk Storage
  • Support Primary Key and multiple Secondary Index
  • Support HASH and RBTREE Index
  • Support Multi-columns Index
  • Support Exact Match, Leftmost Match, Range Match
  • Support C/C++ Struct/Union ORM APIs
  • Support Struct Scheme Auto upgrade/downgrade
  • Support almost all C data types
  • Support rich DEC/HEX/BIN/Bitmap/MAC/IPv4/IPv6/Timestamp formats
  • Support Standard Transaction (begin/commit/rollback)
  • Support WAL for OnDisk storage
  • Dynamic Growing/Shrinking Table
  • Support Multiple Threads and Multiple Processes Access
  • Support Table level read/write lock
  • Super High Performance
  • Very Robust: Process Restart, Process Crash, Power Cycle
  • Very Simple: Simple header and library file
  • Zero Config: no complex config, real out-of-the-box

Use Cases

  • You can use CrossDB RDBMS to Manage Program Runtime Data efficiently.
  • You can use CrossDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CrossDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CrossDB RDBMS to refactor your code conveniently.
  • You can use CrossDB Index to optimize performance without changing your code.

Comments

\ No newline at end of file diff --git a/docs/reference/crossbench/index.html b/docs/reference/crossbench/index.html deleted file mode 100644 index f011654..0000000 --- a/docs/reference/crossbench/index.html +++ /dev/null @@ -1,204 +0,0 @@ - CrossDB Benchmark - CrossDB

CrossDB Benchmark

CrossBench is an utility designed for evaluating performance of embedded databases. The goal of this project is to provide a standart and simple in use instrument for benchmarking, so any database developer or user can reference to or repeat obtained results.

  • Storage modes: On-Disk, In-Memory, On-Ramdisk
  • Benchmarking methods: insert, load, query(top 3 avg), update(top 5 avg), delete
  • Access Sequence: random, sequential
  • Thread: Single thread now, you can bind cpu core to do test
  • Supported databases: crossdb, sqlite3, lmdb, STL map, STL hashmap(unordered_map)

Usage

./crossdb-bench.bin -h
-Usage: {count unit can be k or K (*1000)  m or M (*1000000)}
-  -h                        show this help
-  -s <db storage>           ramdisk | inmem | ondisk, default is ramdisk
-                            shortcut r: ramdisk  m: inmem    d: ondisk
-                            ondisk store db in current folder crossbench
-  -i <insert count>         default 1000000, do insert test, conflict with -l load
-  -r <row count>            open db which already has these rows
-  -q <query count>          default 1000000, do query test
-  -u <update count>         default 1000000, do update test
-  -d <delete count>         default 1000000, do delete test
-  -c <cpu core>             bind cpu core
-  -l <load count>           quick load data, conflict with -i insert
-  -k <key type>             default | hash | tree
-                            shortcut h: hash  t: tree    d: default
-  -S                        sequential, defaut is random
-  -V                        verify driver basic CRUD operations
-  -Q                        quiet mode, only show last result
-  -H                        show header, if -Q is set, then don't show header
-
-Example:
-  default: do ramdisk random mode test, insert 1M rows, query 1M times, update 1M times, delete 1M rows
-     -s m: do inmem random mode test,   insert 1M rows, query 1M times, update 1M times, delete 1M rows
-     -s d: do ondisk random mode test,  insert 1M rows, query 1M times, update 1M times, delete 1M rows
-     -i 500k -q 5m:            do ramdisk random mode test, insert 500K rows, query 5M times, updat 1M times, delete 500k rows
-     -i 10k -q 0 -u 0 -d 0:    do ramdisk random mode test, insert 10K rows
-     -r 10k -q 100k -u 0 -d 0: do ramdisk random mode test, query  100K times
-     -r 10k -q 0 -u 100k -d 0: do ramdisk random mode test, update 100K times
-     -r 10k -q 0 -u 0 -d 10k:  do ramdisk random mode test, delete 10K rows
-     -V:  verify db driver is ok
-

Note

ondisk DB is stored in current folder _benchmarkdb, ramdisk DB is stored in /tmp/_benchmarkdb. If you don't delete all rows in your test, you'd better remember to delete manually or do a simple default test or ./build.sh clean.

Build

Install sqlite3 and lmdb

sudo apt install libsqlite3-dev
-sudo apt install liblmdb-dev
-

Install CrossDB Library

sudo cp libxxxdb.so /usr/lib/
-

Build 

./build.sh
-

Output Example

./crossdb-bench.bin
-CrossDB Database Benchmark
-
-Location: /tmp/_benchmarkdb/CrossDB
-Type: RAMDISK
-Key: Default
-Access: Random
-
-Insert rows: 1000000    Use time: 272466us    TPS: 3670182
-
-Query Test: 1000000
-  Round: 1      Use Time: 231543us      QPS: 4318852
-  Round: 2      Use Time: 244874us      QPS: 4083732
-  Round: 3      Use Time: 240435us      QPS: 4159128
-  Round: 4      Use Time: 223147us      QPS: 4481350
-  Round: 5      Use Time: 221954us      QPS: 4505438
-  Round: 6      Use Time: 243468us      QPS: 4107315
-  Round: 7      Use Time: 240375us      QPS: 4160166
-  Round: 8      Use Time: 240878us      QPS: 4151479
-  Round: 9      Use Time: 229140us      QPS: 4364144
-  Round: 10     Use Time: 237540us      QPS: 4209817
-Update Test: 1000000
-  Round: 1      Use Time: 266616us      TPS: 3750712
-  Round: 2      Use Time: 281096us      TPS: 3557503
-  Round: 3      Use Time: 274047us      TPS: 3649009
-  Round: 4      Use Time: 265904us      TPS: 3760755
-  Round: 5      Use Time: 260426us      TPS: 3839862
-  Round: 6      Use Time: 265945us      TPS: 3760175
-  Round: 7      Use Time: 254995us      TPS: 3921645
-  Round: 8      Use Time: 274223us      TPS: 3646667
-  Round: 9      Use Time: 274394us      TPS: 3644394
-  Round: 10     Use Time: 290548us      TPS: 3441772
-
-Delete: 1000000 Use Time: 401207us      TPS: 2492478
-
-CrossDB Benchmark Result: RAMDISK Access=Random Key=Default Rows=1000000
-              DB            Rows          Insert           Query          Update          Delete
-         CrossDB         1000000         3670182         4450310         3840754         2492478
-

Benchmark Scripts

You can use following scripts to do benchmark test. The parameter is the cpu core to test which is optional, but to make test reliable, you'd better do test or low load system.

./crossdb-benchmark.sh 7
-./sqlite-benchmark.sh 7
-./lmdb-benchmark.sh 7
-./stlmap-benchmark.sh 7
-./stlhmap-benchmark.sh 7
-

Output example 

On-Disk Benchmark Test
-              DB            Rows          Insert           Query          Update          Delete
-         CrossDB            1000          351741        22158982         2441267         2785515
-         CrossDB            1000          651465        22109811         2453643         2865329
-         CrossDB            1000          643500        22131423         2436465         2770083
-         CrossDB           10000         1523229        19490360         2385504         3036744
-         CrossDB           10000         1671402        19781879         2402664         3087372
-         CrossDB           10000         1538224        19753758         2383397         3049710
-         CrossDB          100000         1923409        14145718         2157846         2595178
-         CrossDB          100000         1833483        14079026         2152338         2472615
-         CrossDB          100000         1895123        13783290         2145765         1340213
-         CrossDB         1000000         1054961         6716753         1471987         1721339
-         CrossDB         1000000         1058803         6712856         1464983         1725965
-         CrossDB         1000000         1051711         6729894         1469273         1749135
-         CrossDB        10000000          598117         4020796         1202939         1172451
-         CrossDB        10000000          595118         3992847         1196629         1133848
-         CrossDB        10000000          596664         4025642         1189165         1119912
-         CrossDB        100000000         429711         3052691          924709          514820
-         CrossDB        100000000         394050         2984995         1081168          552174
-         CrossDB        100000000         372579         3002053         1084742          630738
-On-RamDisk Benchmark Test
-              DB            Rows          Insert           Query          Update          Delete
-         CrossDB            1000         3134796        22019752        15952443         5076142
-         CrossDB            1000         1577287        22070616        16006938         5208333
-         CrossDB            1000         1623376        22059753        15972537         5128205
-         CrossDB           10000         3095975        19691196        14472099         6165228
-         CrossDB           10000         3454231        19662306        14318556         6265664
-         CrossDB           10000         2882675        19533775        13952115         6042296
-         CrossDB          100000         3547357        14193827        10452799         4989770
-         CrossDB          100000         3598934        13253040        10448443         5093205
-         CrossDB          100000         3744196        14170861        10174198         5261496
-         CrossDB         1000000         3740750         6740538         5310981         3029302
-         CrossDB         1000000         3708868         6753996         5488757         3017146
-         CrossDB         1000000         3682956         6748722         5491734         2986920
-         CrossDB        10000000         1936138         4006575         3570695         1803817
-         CrossDB        10000000         2331095         4023692         3568680         1858624
-         CrossDB        10000000         2271060         4018655         3566541         2015747
-         CrossDB        100000000        1466629         3007293         2734737          802871
-         CrossDB        100000000        1467744         2967245         2714797          815151
-         CrossDB        100000000        1442414         3004026         2734427          747224
-In-Memory Benchmark Test
-              DB            Rows          Insert           Query          Update          Delete
-         CrossDB            1000         4016064        21937928        15225355         7812500
-         CrossDB            1000         1838235        22083004        15972435         7575757
-         CrossDB            1000         1461988        22109461        16017004         8333333
-         CrossDB           10000         5356186        19717491        14388566         7163323
-         CrossDB           10000         5254860        19530617        14567100         7385524
-         CrossDB           10000         3285151        19605321        14355210         7047216
-         CrossDB          100000         4159733        13979051        10180695         5471656
-         CrossDB          100000         3981367        14143811        10259367         5474052
-         CrossDB          100000         4039098        13841855        10042771         5144562
-         CrossDB         1000000         3907868         6742926         5475764         3044659
-         CrossDB         1000000         3972399         6741476         5378830         3184835
-         CrossDB         1000000         3963362         6761225         5484965         3167213
-         CrossDB        10000000         2489824         3997149         3556671         2038462
-         CrossDB        10000000         2480423         4013349         3555828         2070516
-         CrossDB        10000000         2486211         3913256         3449931         1956951
-         CrossDB        100000000        1696429         2807183         2546983         1468625
-         CrossDB        100000000        1789488         2903683         2613381         1480978
-         CrossDB        100000000        1783067         2886531         2627970         1492552
-

New Driver

To make a new xxxdb driver, refer the driver template below.

#include "benchmark.h"
-
-const char* xxxdb_create (char *dbname, dbtype_e type, keytype_e key, uint32_t flags) 
-{
-    // credata database or table, Primary Key: prefix+mask
-}
-
-void xxxdb_close ()
-{
-    // close database
-}
-
-void xxxdb_begin ()
-{
-    // begin transaction
-}
-
-void xxxdb_commit ()
-{
-    // commit transaction
-}
-
-void xxxdb_insert (route_t *pRoute)
-{
-    // insert route
-}
-
-bool xxxdb_query (route_t *pRoute)
-{
-    // query route by Primary Key
-}
-
-void xxxdb_update (route_t *pRoute)
-{
-    // update route set birth by Primary Key
-}
-
-void xxxdb_delete (route_t *pRoute)
-{
-    // delete route by Primary Key
-}
-
-#define db_name     "xxxDB"
-#define db_create   xxxdb_create
-#define db_close    xxxdb_close
-#define db_insert   xxxdb_insert
-#define db_query    xxxdb_query
-#define db_update   xxxdb_update
-#define db_delete   xxxdb_delete
-#define db_begin    xxxdb_begin
-#define db_commit   xxxdb_commit
-
-#include "benchmark.c"
-

Verify the driver can work 

./xxxdb-bench.bin -V
------- Verify Driver ------
-Insert route
-Query route
-10.1.1.0/24->10.1.1.254 intf: eth1 metric: 1 birth: 1687848698 flags: 0x0
-Update route
-Query route
-10.1.1.0/24->10.1.1.254 intf: eth1 metric: 1 birth: 1687848708 flags: 0x0
-Delete route
-Query route
-crossdb error: status = -4, no row found
-10.1.1.0/24
------- Verify Done ------
-

\ No newline at end of file diff --git a/docs/reference/crossdb-cli/index.html b/docs/reference/crossdb-cli/index.html deleted file mode 100644 index 3bce4d5..0000000 --- a/docs/reference/crossdb-cli/index.html +++ /dev/null @@ -1,157 +0,0 @@ - CrossDB CLI - CrossDB

CrossDB CLI

The crossdb-cli CrossDB Command Line Interface (CLI) is the simplest way for users to manipulate and interact with CrossDB.

Installation

There is no need for additional installation steps to install CrossDB CLI as it is already included in package automatically.

Execution

To access the CrossDB CLI, you can execute crossdb-cli command-line utility from a terminal or double click in windows. 

./crossdb-cli
-   _____                   _____  ____      _
-  / ____|                 |  __ \|  _ \   _| |_
- | |     _ __ ___  ___ ___| |  | | |_) | |_   _|
- | |    | '__/ _ \/ __/ __| |  | |  _ <    |_|
- | |____| | | (_) \__ \__ \ |__| | |_) |
-  \_____|_|  \___/|___/___/_____/|____/ crossdb.org
-
-
-CrossDB 0.5.0  DB version 0.1  Little Endian  64bit
-
-============ Welcome to CrossDB Shell ============
-<help>: Help Info       <F1>: Shortcuts
-<exit>: Exit shell      <TAB>: Auto completion
-
-CrossDB>
-
After entering the CrossDB CLI, you can execute various SQL commands.

CrossDB CLI tips

  • Press TAB key to get auto completion
  • Use up and down keys to iterate the history of commands entered
  • Use F1 to get all shortcuts
  • ./crossdb-cli /tmp/_benchmarkdb/CrossDB open DB directly
  • Enter open /tmp/_benchmarkdb/CrossDB to oepn DB interactly
  • Enter show d to show all opened databases
  • Enter show t to show all tables in current database
  • Enter use <db> to switch currenbt database
  • Enter exit to exit CrossDB CLI
  • Append >> at end to redirect output to file

Command List

Press TAB key to show it 

CrossDB>
-select      Select records from TABLE
-insert      Insert a record to TABLE
-replace     Replace a record to TABLE (update if exists else insert)
-update      Update records in TABLE
-delete      Delete records from TABLE
-begin       Begin Transaction
-commit      Commit Transaction
-rollback    Rollback Transaction
-create      Create INDEX, TABLE, DATABASE
-drop        Drop INDEX, TABLE, DATABASE [!!! Caution !!!]
-show        Show tables, databases, indexes, etc
-use         Switch DATABASE
-open        Open DATABASE
-close       Close DATABASE
-history     Show history
-exit        Exit shell
-

Database Command

Open Databases

./crossdb-cli /tmp/_benchmarkdb/CrossDB
-
-CrossDB> open /tmp/_benchmarkdb/CrossDB
-

Switch Databases

CrossDB> use CrossDB
-Current Datatbase is changed to CrossDB
-

Show Command

Show Databases

CrossDB> show d
-ID  Name     Tables  Type     Share  Size  Path
-==  =======  ======  =======  =====  ====  =========================
-1   CrossDB  1       RamDisk  F      5792  /tmp/_benchmarkdb/CrossDB
-
-Current Database is CrossDB ("use <dbname>" to switch db)
-

Show Tables

CrossDB> show t
-ID  Name   RowCount  RowLimit  RowSize  Columns  Indexes  TblSize
-==  =====  ========  ========  =======  =======  =======  =======
-1   route  0         0         48       7        1        5672
-

Show Tables Indexes

CrossDB> show indexes
-ID  Table  Index    Columns      Primary  Unique  Type  RowNum  NodeNum  SlotUsed  Size
-==  =====  =======  ===========  =======  ======  ====  ======  =======  ========  ====
-1   route  PRIMARY  prefix,mask  T        T       HASH  0       0        0         1216
-

Show Tables Status

CrossDB> show status
-ID  Name   RowCount  PoolCount  Cursor  TblSize  DatSize  IdxSize
-==  =====  ========  =========  ======  =======  =======  =======
-1   route  0         8          0       5672     784      1216
-

Show Tables Counters

CrossDB> show counters
-ID  Name   Scan  Insert   Update    Delete   InsertFail  UpdateFail  DeleteFail
-==  =====  ====  =======  ========  =======  ==========  ==========  ==========
-1   route  0     1000000  10000000  1000000  0           0           0
-

Show table schema

CrossDB> show create table route
-create table `route` (
-  `prefix`       UINT(4,0)      format IPv4,
-  `mask`         UINT(1,4),
-  `nexthop`      UINT(4,8)      format IPv4,
-  `metric`       UINT(1,12),
-  `intf`         CHAR(16,13),
-  `birth`        UINT(8,32)     format TIMESTAMP,
-  `flags`        UINT(4,40)     format HEX,
-
-  PRIMARY KEY (prefix,mask)
-) rec_size=48;
-

Select Command

select < * | column1,columnm2,...> from <table> 
-    [ where columnA=valueA and columnB=valueB and ... ]
-    [ order by columnX,columnY,... [desc|asc] ]
-    [ limit <n> ] [ offset <m> ] | [limit offset,row_count]
-

Where operators: =, !=, >, >=, <, <=

Note

Default select * will only show none zero fields, select ** can show all fields.
One field can only apper twice.
Only AND is supported

CrossDB> select * from route limit 10
-ID  prefix     mask  nexthop     metric  intf  birth                flags
-==  =========  ====  ==========  ======  ====  ===================  =====
-1   10.1.1.2   24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0
-2   10.1.1.55  24    10.1.1.254  1       eth1  2023-10-20T11:01:06  0x0
-3   10.1.1.18  24    10.1.1.254  1       eth1  2023-10-20T11:01:01  0x0
-4   10.1.1.90  24    10.1.1.254  1       eth1  2023-10-20T10:59:08  0x0
-5   10.1.1.37  24    10.1.1.254  1       eth1  2023-10-20T11:01:12  0x0
-6   10.1.1.27  24    10.1.1.254  1       eth1  2023-10-20T11:01:33  0x0
-7   10.1.1.61  24    10.1.1.254  1       eth1  2023-10-20T11:00:52  0x0
-8   10.1.1.95  24    10.1.1.254  1       eth1  2023-10-20T11:00:29  0x0
-9   10.1.1.46  24    10.1.1.254  1       eth1  2023-10-20T10:57:50  0x0
-10  10.1.1.75  24    10.1.1.254  1       eth1  2023-10-20T11:02:18  0x0
-
-Qualified Rows: 100     Select Rows: 10    Use time 34us QPS 294117
-
CrossDB> select * from route where prefix=10.1.1.2
-ID  prefix    mask  nexthop     metric  intf  birth                flags
-==  ========  ====  ==========  ======  ====  ===================  =====
-1   10.1.1.2  24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0
-
-Qualified Rows: 1       Select Rows: 1    Use time 8us QPS 125000
-
CrossDB> select * from route where prefix>=10.1.1.10 and prefix<=10.1.1.20
-ID  prefix     mask  nexthop     metric  intf  birth                flags
-==  =========  ====  ==========  ======  ====  ===================  =====
-1   10.1.1.18  24    10.1.1.254  1       eth1  2023-10-20T11:01:01  0x0
-2   10.1.1.20  24    10.1.1.254  1       eth1  2023-10-20T10:59:22  0x0
-3   10.1.1.14  24    10.1.1.254  1       eth1  2023-10-20T11:02:04  0x0
-4   10.1.1.11  24    10.1.1.254  1       eth1  2023-10-20T11:01:50  0x0
-5   10.1.1.12  24    10.1.1.254  1       eth1  2023-10-20T11:01:44  0x0
-6   10.1.1.19  24    10.1.1.254  1       eth1  2023-10-20T11:01:09  0x0
-7   10.1.1.16  24    10.1.1.254  1       eth1  2023-10-20T11:00:56  0x0
-8   10.1.1.17  24    10.1.1.254  1       eth1  2023-10-20T11:02:32  0x0
-9   10.1.1.15  24    10.1.1.254  1       eth1  2023-10-20T10:58:50  0x0
-10  10.1.1.10  24    10.1.1.254  1       eth1  2023-10-20T10:59:38  0x0
-11  10.1.1.13  24    10.1.1.254  1       eth1  2023-10-20T11:02:29  0x0
-
CrossDB> select * from route order by prefix
-ID   prefix     mask  nexthop     metric  intf  birth                flags
-===  =========  ====  ==========  ======  ====  ===================  =====
-1    10.1.1.0   24    10.1.1.254  1       eth1  2023-10-20T11:00:23  0x0
-2    10.1.1.1   24    10.1.1.254  1       eth1  2023-10-20T11:02:36  0x0
-3    10.1.1.2   24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0
-4    10.1.1.3   24    10.1.1.254  1       eth1  2023-10-20T11:02:21  0x0
-5    10.1.1.4   24    10.1.1.254  1       eth1  2023-10-20T11:02:52  0x0
-6    10.1.1.5   24    10.1.1.254  1       eth1  2023-10-20T11:01:36  0x0
-7    10.1.1.6   24    10.1.1.254  1       eth1  2023-10-20T11:02:25  0x0
-8    10.1.1.7   24    10.1.1.254  1       eth1  2023-10-20T11:00:43  0x0
-9    10.1.1.8   24    10.1.1.254  1       eth1  2023-10-20T11:01:20  0x0
-

DML Commands

Warning

Database is accessed exclusively by single process with best performance. So if process is running and accessing the DB, please don't do the DML commands, otherwise the table may be corrupted. You can update fields not used in index, but be careful to do it.

Insert Row

insert into route set prefix=1.1.1.1 mask=24 nexthop=1.1.1.254 metric=5 intf='eth1'
-

Note

Where clause follows select syntax.

Update Row

update route set metric=5 where prefix=1.1.1.1
-

Note

Where clause follows select syntax.

Delete Row

delete from route where prefix=1.1.1.1
-

Note

Where clause follows select syntax.

Comments

\ No newline at end of file diff --git a/faq/index.html b/faq/index.html new file mode 100755 index 0000000..2248d65 --- /dev/null +++ b/faq/index.html @@ -0,0 +1,31 @@ + FAQ - CrossDB

FAQ

Why is CrossDB so fast?

  • Use memory map to access DB data directly (so your DB must less than the main memory to get highest speed).
  • Use high-performance hash as the main index, and use the super fast wyhash as the hash function.
  • Hand-written SQL parser which is more than 10X faster than Flex/Bison parser tools.
  • High-performance read-write lock and multi-core optimization.
  • Execute multiple SQL statements in one API, which will imporve client-server DB performance dramatically.
  • High-efficient client-server protocols.
  • Avoid memory allocation/free as much as possible.
  • High-performance implementation code

Comments

\ No newline at end of file diff --git a/get-started/bench/index.html b/get-started/bench/index.html new file mode 100755 index 0000000..b53ba42 --- /dev/null +++ b/get-started/bench/index.html @@ -0,0 +1,124 @@ + Example and Bench - CrossDB

Example and Bench

Example

crossdb$ make example
+=== Select all 7 rows
+id=1 name='jack' age=10 class='3-1' score=90.000000 info=''
+id=2 name='tom' age=11 class='2-5' score=91.000000 info=''
+id=3 name='jack' age=11 class='1-6' score=92.000000 info=''
+id=4 name='rose' age=10 class='4-2' score=90.000000 info=''
+id=5 name='tim' age=10 class='3-1' score=95.000000 info=''
+id=6 name='Tony' age=10 class='3-1' score=95.000000 info='He is a boy.
+He likes playing football.
+We all like him!'
+id=7 name='Wendy' age=10 class='3-1' score=95.000000 info='She is a girl.
+She likes cooking.
+We all love her!'
+
+=== Update age = 9 for id = 2
+select 1 rows
+id=2 name='tom' age=9 class='2-5' score=91.000000
+id=2 name='tom' age=9 class='2-5' score=91.000000
+id=2 name='tom' age=9 class='2-5' score=91.000000
+
+=== Delete id = 3
+select 0 rows
+=== select 1 rows
+COUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667
+COUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667
+COUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667
+
+=== Muti-Statements
+COUNT(*)=6
+id=2 name='tom'
+   _____                   _____  ____      _
+  / ____|                 |  __ \|  _ \   _| |_
+ | |     _ __ ___  ___ ___| |  | | |_) | |_   _|
+ | |    | '__/ _ \/ __/ __| |  | |  _ <    |_|
+ | |____| | | (_) \__ \__ \ |__| | |_) |  0.6.0
+  \_____|_|  \___/|___/___/_____/|____/ crossdb.org
+
+============ Welcome to CrossDB Shell ============
+<help>: Help Info       <F1>: Shortcuts
+<exit>: Exit shell      <TAB>: Auto completion
+
+XDB> exit
+

Bench Test

Test with 1000 rows

crossdb/bench/c$ ./a.out 1000
+********************** INSERT 1000 with SQL **********************
+Use time 763ns, QPS 1310615
+********************** Random LKUP 5000000 with SQL **********************
+Use time 951987ns, QPS 5252172
+Use time 921033ns, QPS 5428687
+Use time 915605ns, QPS 5460870
+Use time 926288ns, QPS 5397889
+********************** UPDATE 1000 with SQL **********************
+Use time 760ns, QPS 1315789
+********************** DELETE 1000 with SQL **********************
+Use time 438ns, QPS 2283105
+
+********************** INSERT 1000 with SQL **********************
+Use time 687ns, QPS 1455604
+********************** Random LKUP 10000000 with Prepared STMT **********************
+Use time 294717ns, QPS 33930855
+Use time 297268ns, QPS 33639678
+Use time 298728ns, QPS 33475268
+Use time 295278ns, QPS 33866390
+********************** UPDATE 1000 with Prepared SMTT **********************
+Use time 576ns, QPS 1736111
+********************** DELETE 1000 with Prepared SQL **********************
+Use time 252ns, QPS 3968253
+

Test with 1000,1000 rows

crossdb$ make bench
+or
+crossdb/bench/c$ ./a.out
+
+********************** INSERT 1000000 with SQL **********************
+Use time 539342ns, QPS 1854111
+********************** Random LKUP 1000000 with SQL **********************
+Use time 336044ns, QPS 2975800
+Use time 333927ns, QPS 2994666
+Use time 329699ns, QPS 3033069
+Use time 331632ns, QPS 3015390
+********************** UPDATE 1000000 with SQL **********************
+Use time 579945ns, QPS 1724301
+********************** DELETE 1000000 with SQL **********************
+Use time 405004ns, QPS 2469111
+
+********************** INSERT 1000000 with SQL **********************
+Use time 527486ns, QPS 1895784
+********************** Random LKUP 10000000 with Prepared STMT **********************
+Use time 1678936ns, QPS 5956153
+Use time 1668971ns, QPS 5991715
+Use time 1670993ns, QPS 5984465
+Use time 1652815ns, QPS 6050283
+********************** UPDATE 1000000 with Prepared SMTT **********************
+Use time 381320ns, QPS 2622469
+********************** DELETE 1000000 with Prepared SQL **********************
+Use time 232181ns, QPS 4306984
+

Comments

\ No newline at end of file diff --git a/get-started/install/index.html b/get-started/install/index.html new file mode 100755 index 0000000..deb0fa9 --- /dev/null +++ b/get-started/install/index.html @@ -0,0 +1,47 @@ + Build and Install - CrossDB

Build and Install

Build

crossdb$ make
+make build                Build crossdb library and tool
+make debug                Build crossdb library and tool with debug
+make run                  Run crossdb tool
+make clean                Clean build result
+make install              Install crossdb(lib&tool&header) to Linux & FreeBSD
+make uninstall            Uninstall crossdb from Linux & FreeBSD
+make installmacos         Install crossdb(lib&tool&header) to MacOS
+make uninstallmacos       Uninstall crossdb from MacOS
+make example              Build and run example (need to install crossdb first)
+make bench                Build and run bench test (need to install crossdb first)
+

Linux/FreeBSD

make build
+sudo make install
+

MacOS

make build
+sudo make installmacos
+

Windows

You need to install MINGW64 to build. Then set gcc path to system environment variables Path and make sure gcc can run.

build
+

Comments

\ No newline at end of file diff --git a/get-started/tutorial/index.html b/get-started/tutorial/index.html new file mode 100755 index 0000000..768617f --- /dev/null +++ b/get-started/tutorial/index.html @@ -0,0 +1,179 @@ + Turorial - CrossDB

Turorial

  • Open conection and create default memory database
xdb_res_t   *pRes;
+xdb_row_t   *pRow;
+
+xdb_conn_t  *pConn = xdb_open (":memory:");
+
  • Create Table
pRes = xdb_exec (pConn, "CREATE TABLE student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(255))");
+XDB_CHECK(pRes, printf ("Can't create table student\n"); goto error;);
+pRes = xdb_exec (pConn, "CREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(255), INDEX (name))");
+XDB_CHECK(pRes, printf ("Can't create table teacher\n"); goto error;);
+pRes = xdb_exec (pConn, "CREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name))");
+XDB_CHECK(pRes, printf ("Can't create table book\n"); goto error;);
+
  • Insert Rows
pRes = xdb_exec (pConn, "INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)");
+XDB_CHECK(pRes, printf ("Can't insert table student\n"); goto error;);
+pRes = xdb_pexec (pConn, "INSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')", "He is a boy.\nHe likes playing football.\nWe all like him!");
+XDB_CHECK(pRes, printf ("Can't insert table student\n"); goto error;);
+pRes = xdb_pexec (pConn, "INSERT INTO student (id,name,age,class,score,info) VALUES (7,'Wendy',10,'3-1',95,'%s')", "She is a girl.\nShe likes cooking.\nWe all love her!");
+XDB_CHECK(pRes, printf ("Can't insert table student\n"); goto error;);
+pRes = xdb_exec (pConn, "INSERT INTO teacher (id,name,age) VALUES (1,'Tomas',40),(2,'Steven',50),(3,'Bill',31),(4,'Lucy',29)");
+XDB_CHECK(pRes, printf ("Can't insert table teacher\n"); goto error;);
+pRes = xdb_exec (pConn, "INSERT INTO book (id,name,author,count) VALUES (1,'Romeo and Juliet','Shakespeare',10),(2,'Pride and Prejudice','Austen',5),(3,'Great Expectations','Dickens',8),(4,'Sorrows of Young Werther','Von Goethe',4)");
+XDB_CHECK(pRes, printf ("Can't insert table book\n"); goto error;);
+
  • Select Rows
pRes = xdb_exec (pConn, "SELECT * from student");
+printf ("=== Select all %d rows\n", (int)pRes->row_count);
+while (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+    printf ("id=%d name='%s' age=%d class='%s' score=%f\n", 
+        xdb_column_int (pRes->col_meta, pRow, 0), 
+        xdb_column_str (pRes->col_meta, pRow, 1), 
+        xdb_column_int (pRes->col_meta, pRow, 2), 
+        xdb_column_str (pRes->col_meta, pRow, 3), 
+        xdb_column_float(pRes->col_meta, pRow, 4));
+    printf ("id=%d name='%s' age=%d class='%s' score=%f\n", 
+        *(int*)pRow[0], 
+        (char*)pRow[1], 
+        *(int*)pRow[2], 
+        (char*)pRow[3], 
+        *(float*)pRow[4]);
+}
+xdb_free_result (pRes);
+
  • Update Rows
printf ("\n=== Update age = 9 for id = 2\n");
+pRes = xdb_exec (pConn, "UPDATE student set age=9 WHERE id = 2");
+XDB_CHECK(pRes, printf ("Can't update id=%d\n",2); goto error;);
+
+pRes = xdb_exec (pConn, "SELECT id,name,age,class,score from student WHERE id = 2");
+printf ("select %d rows\n", (int)pRes->row_count);
+while (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+
  • Delete Rows
printf ("\n=== Delete id = 3\n");
+pRes = xdb_exec (pConn, "DELETE FROM student WHERE id = 3");
+XDB_CHECK(pRes, printf ("Can't delete id=%d\n",3); goto error;);
+
+pRes = xdb_exec (pConn, "SELECT * from student WHERE id = 3");
+printf ("select %d rows\n", (int)pRes->row_count);
+while (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+
  • Aggregation function
printf ("\n=== AGG COUNT,MIN,MAX,SUM,AVG\n");
+pRes = xdb_exec (pConn, "SELECT COUNT(*),MIN(score),MAX(score),SUM(score),AVG(score) FROM student");
+printf ("=== select %d rows\n", (int)pRes->row_count);
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+    printf ("COUNT(*)=%d MIN(score)=%f MAX(score)=%f SUM(score)=%f AVG(score)=%f\n", 
+        xdb_column_int (pRes->col_meta, pRow, 0), 
+        xdb_column_double(pRes->col_meta, pRow, 1), 
+        xdb_column_double(pRes->col_meta, pRow, 2), 
+        xdb_column_double(pRes->col_meta, pRow, 3), 
+        xdb_column_double(pRes->col_meta, pRow, 4));
+    printf ("COUNT(*)=%d MIN(score)=%f MAX(score)=%f SUM(score)=%f AVG(score)=%f\n", 
+        (int)*(int64_t*)pRow[0], 
+        *(float*)pRow[1], 
+        *(float*)pRow[2], 
+        *(double*)pRow[3], 
+        *(double*)pRow[4]);
+}
+xdb_free_result (pRes);
+
  • Transaction Rollback
printf ("\n=== Rollback\n");
+xdb_begin (pConn);
+pRes = xdb_exec (pConn, "UPDATE student set age=15 WHERE id = 2");
+pRes = xdb_exec (pConn, "SELECT id,name,age from student WHERE id = 2");
+printf ("select %d rows\n", (int)pRes->row_count);
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+printf ("-- rollback\n");
+xdb_rollback (pConn);
+pRes = xdb_exec (pConn, "SELECT id,name,age from student WHERE id = 2");
+printf ("select %d rows\n", (int)pRes->row_count);
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+
  • Transaction Commit
printf ("\n=== Commit\n");
+xdb_begin (pConn);
+pRes = xdb_exec (pConn, "UPDATE student set age=15 WHERE id = 2");
+pRes = xdb_exec (pConn, "SELECT * from student WHERE id = 2");
+printf ("select %d rows\n", (int)pRes->row_count);
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+printf ("-- commit\n");
+xdb_commit (pConn);
+pRes = xdb_exec (pConn, "SELECT * from student WHERE id = 2");
+printf ("select %d rows\n", (int)pRes->row_count);
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+
  • Multi-Statements
printf ("\n=== Muti-Statements\n");
+pRes = xdb_exec (pConn, "SELECT COUNT(*) FROM student; SELECT id,name FROM student WHERE id=2");
+// count(*) 
+if (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->col_meta, pRow, 0);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+// select
+pRes = xdb_next_result (pConn);
+if (NULL != pRes) {
+    if (NULL != (pRow = xdb_fetch_row (pRes))) {
+        xdb_print_row (pRes->col_meta, pRow, 0);
+        printf ("\n");
+    }
+    xdb_free_result (pRes);
+}
+
+xdb_exec (pConn, "SHELL");
+

Comments

\ No newline at end of file diff --git a/glossary/index.html b/glossary/index.html new file mode 100755 index 0000000..1729ea1 --- /dev/null +++ b/glossary/index.html @@ -0,0 +1,31 @@ + Glossary - CrossDB

Glossary

ACID

ACID refers to the four key properties of a transaction: atomicity, consistency, isolation, and durability. Each of these properties is described below.

Atomicity means that either all the changes of an operation are performed, or none of them are.

Consistency means that transactions always bring the database from one consistent state to another.

Isolation means that a transaction in process is invisible to other transactions until it completes. This allows concurrent transactions to read and write data without sacrificing consistency.

Durability means that once a transaction is committed, it remains committed even in the event of a system failure.

CRUD

Acronym for “create, read, update, delete”, a common sequence of operations in database applications. Often denotes a class of applications with relatively simple database usage (basic DDL, DML and query statements in SQL) that can be implemented quickly in any language.

DDL

Data definition language, a set of SQL statements for manipulating the database itself rather than individual table rows. Includes all forms of the CREATE, ALTER, and DROP statements. DDL statements automatically commit the current transaction; they cannot be rolled back.

DML

Data manipulation language, a set of SQL statements for performing SELECT, INSERT, UPDATE, and DELETE operations.

Hash Index

A type of index intended for queries that use equality operators, rather than range operators such as greater-than or BETWEEN.

IMDB

IMDB(In-Memory database) is a type of database system that maintains data in memory, to avoid overhead due to disk I/O and translation between disk blocks and memory areas.

Index

A data structure that provides a fast lookup capability for rows of a table, typically by forming a tree structure (B-tree) representing all the values of a particular column or set of columns.

INFORMATION_SCHEMA

The name of the database that provides a query interface to the CrpssDB data dictionary. (This name is defined by the ANSI SQL standard.) To examine information (metadata) about the database, you can query tables such as INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.COLUMNS, rather than using SHOW commands that produce unstructured output.

MVCC

MVCC(Multi-Version Concurrency Control) is a concurrency control mechanism in many databases. It processes the memory read by transactions to achieve concurrent access to DB, thereby avoiding blocking caused by conflicts between concurrent reads and writes.

OLTP

Acronym for “Online Transaction Processing”. A database system, or a database application, that runs a workload with many transactions, with frequent writes as well as reads, typically affecting small amounts of data at a time.

SQL

The Structured Query Language that is standard for performing database operations. Often divided into the categories DDL, DML, and queries.

Comments

\ No newline at end of file diff --git a/images/benchmark/crossdb-vs-lmdb-ondisk-large.png b/images/benchmark/crossdb-vs-lmdb-ondisk-large.png deleted file mode 100644 index f6a278a..0000000 Binary files a/images/benchmark/crossdb-vs-lmdb-ondisk-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-lmdb-ondisk-small.png b/images/benchmark/crossdb-vs-lmdb-ondisk-small.png deleted file mode 100644 index 973e478..0000000 Binary files a/images/benchmark/crossdb-vs-lmdb-ondisk-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-lmdb-ramdisk-large.png b/images/benchmark/crossdb-vs-lmdb-ramdisk-large.png deleted file mode 100644 index 9dfab01..0000000 Binary files a/images/benchmark/crossdb-vs-lmdb-ramdisk-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-lmdb-ramdisk-small.png b/images/benchmark/crossdb-vs-lmdb-ramdisk-small.png deleted file mode 100644 index f36de15..0000000 Binary files a/images/benchmark/crossdb-vs-lmdb-ramdisk-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-inmem-large.png b/images/benchmark/crossdb-vs-sqlite-inmem-large.png deleted file mode 100644 index 288e9ff..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-inmem-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-inmem-small.png b/images/benchmark/crossdb-vs-sqlite-inmem-small.png deleted file mode 100644 index fb1afb0..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-inmem-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-ondisk-large.png b/images/benchmark/crossdb-vs-sqlite-ondisk-large.png deleted file mode 100644 index 03779cb..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-ondisk-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-ondisk-small.png b/images/benchmark/crossdb-vs-sqlite-ondisk-small.png deleted file mode 100644 index 19765fd..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-ondisk-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-ramdisk-large.png b/images/benchmark/crossdb-vs-sqlite-ramdisk-large.png deleted file mode 100644 index f68fa6f..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-ramdisk-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-sqlite-ramdisk-small.png b/images/benchmark/crossdb-vs-sqlite-ramdisk-small.png deleted file mode 100644 index 54af7d9..0000000 Binary files a/images/benchmark/crossdb-vs-sqlite-ramdisk-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-stlhmap-large.png b/images/benchmark/crossdb-vs-stlhmap-large.png deleted file mode 100644 index 423bf3f..0000000 Binary files a/images/benchmark/crossdb-vs-stlhmap-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-stlhmap-small.png b/images/benchmark/crossdb-vs-stlhmap-small.png deleted file mode 100644 index 79a6359..0000000 Binary files a/images/benchmark/crossdb-vs-stlhmap-small.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-stlmap-large.png b/images/benchmark/crossdb-vs-stlmap-large.png deleted file mode 100644 index 6a7da62..0000000 Binary files a/images/benchmark/crossdb-vs-stlmap-large.png and /dev/null differ diff --git a/images/benchmark/crossdb-vs-stlmap-small.png b/images/benchmark/crossdb-vs-stlmap-small.png deleted file mode 100644 index 331619d..0000000 Binary files a/images/benchmark/crossdb-vs-stlmap-small.png and /dev/null differ diff --git a/images/crossdb-cli.gif b/images/crossdb-cli.gif deleted file mode 100644 index 1847116..0000000 Binary files a/images/crossdb-cli.gif and /dev/null differ diff --git a/images/crossdb-model.png b/images/crossdb-model.png deleted file mode 100644 index 6a71aa7..0000000 Binary files a/images/crossdb-model.png and /dev/null differ diff --git a/index.html b/index.html old mode 100644 new mode 100755 index 0bb1462..f7da2ab --- a/index.html +++ b/index.html @@ -1,4 +1,8 @@ - CrossDB

CrossDB
The Fastest Embedded Database in the world ✨

Quick Learn🧭 Benchmark 📜 Download 💾

🌌 RDBMS Model

Follows standard RDBMS model: DB, Table, Column, Row, Index, Cursor, Transaction, etc.

⛽ Storage Mode

Supports On-Disk database, In-Memory database and optimized On-RamDisk database.

🔱 Transaction

Support the ACID (Atomic, Consistent, Isolated and Durable) principles.

🚊 C Struct ORM

Supports using C Struct to access DB directly. Almost all C data types are supported.

🚀 Native APIs

Well desgined high Performance Native APIs to define and manipulate DB.

💮 Auto-Upgrade

Change your struct, existing table will be upgraded automatically for you.

♻️ ISSU

Put all your program ISSU data into CrossDB, then you get In-Service Software Upgrade feature easily.

🌄 Out-of-the-Box

No complex configuration or provisioning, single header file and library. Very easy to learn and use.

CrossDB Model

CrossDB CLI

Learn CrossDB in 5 Minutes

 1
+ CrossDB      
 
 
  
 
  
 
   
 
 
 
 
 
  
 
 
 
CrossDB
Super High-performance Embedded and Server RDBMS✨
 
 Quick Learn🧭 Benchmark 📜  
 
 
 
 
     
  •  
    🌌 Simple
     
     
    CrossDB is easy to install and deploy. It has zero external dependencies and runs in-process in its host application or as a single binary with tiny footprint.
     
    •  
  
     
  •  
    ♻️ Portable
     
     
    CrossDB runs on Linux, macOS, Windows, BSD and all popular hardware architectures. It has idiomatic client APIs for major programming languages.
     
     Read more
     
    •  
  •  
    🚀 Super Fast
     
     
    The hand-writing SQL parser and memory-oriented design architecture make the database run SQL at blazing speed.
     
    •  
  •  
    ⛽ Hybrid Storage Mode
     
     
    CrossDB supports On-Disk database and In-Memory database(IMDB). It can also support Hybrid mode with some tables On-Disk and some In-Memory.
     
    •  
  •  
    🔱 MySQL-like SQL
     
     
    Most SQL commands are similar with MySQL.
     
     Read more
     
    •  
  •  
    💮 Server Mode
     
     
    Besides the Embedded RDBMS mode, CrossDB can also run in Embedded Server mode or run CrossDB as Standalone RDBMS server.
     
    •  
 
 
 
 
 
 1
  2
  3
  4
@@ -11,209 +15,49 @@
 11
 12
 13
-14
-15
-16
-17
-18
-19
-20
-21
-22
typedef struct {
-    uint32_t            prefix;
-    uint8_t             mask;
-    uint32_t            nexthop;
-    uint8_t             metric;
-    char                intf[16];
-    uint32_t            birth;
-    uint32_t            flags;
-} route_t;
-
-#undef  CROSS_STRUCT_NAME
-#define CROSS_STRUCT_NAME   route_t
-cross_field_t   route_schema[] = {
-    CROSS_FIELD (prefix,    UINT,   IPv4, 0),
-    CROSS_FIELD (mask,      UINT,   DFT,  0),
-    CROSS_FIELD (nexthop,   UINT,   IPv4, 0),
-    CROSS_FIELD (metric,    UINT,   DFT,  0),
-    CROSS_FIELD (intf,      CHAR,   DFT,  0),
-    CROSS_FIELD (birth,     UINT,   TS,   0),
-    CROSS_FIELD (flags,     UINT,   HEX,  0),
-    CROSS_END (route_t)
-};
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
-18
-19
#define CHECK(ret,str)      if (ret < 0) {  printf (str": %s\n", cross_errMsg(ret)); return -1; }
-
-cross_db_h      hDb;
-cross_tbl_h     hRtTbl;
-cross_ret       ret;
-route_t         route;  
-cross_rowid     count;
-
-// Create database
-ret = cross_dbCreate (&hDb, "db_data/example", 0);
-CHECK (ret, "Failed to create db: example");
-
-// Create table: route (Primary Key: prefix,mask)
-ret = cross_dbTblCreate (hDb, &hRtTbl, "route", route_schema, "prefix,mask", 0);
-CHECK (ret, "Failed to create table: route table");
-
-// Create index on nexthop: idx_nexthop
-ret = cross_dbIdxCreate (hRtTbl, "idx_nexthop", "nexthop", 0);
-CHECK (ret, "Failed to create index: idx_nexthop");
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
#define IP4ADDR(a,b,c,d)    ((a)<<24|(b)<<16|(c)<<8|(d))
-
-// Insert route 192.168.1.0/24->192.168.1.254
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.nexthop   = IP4ADDR(192,168,1,254);
-route.metric    = 1;
-route.flags     = 0;
-strcpy (route.intf, "eth1");
-route.birth     = time (NULL);
-ret = cross_dbInsertRow (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to insert route 192.168.1.0/24");
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
-14
-15
-16
-17
#define IP4STR(ip)              ip>>24,(ip>>16)&0xff,(ip>>8)&0xff,ip&0xff
-
-// Get single route 192.168.1.0/24 by Primary Key
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0); 
-CHECK (ret, "Failed to get route 192.168.1.0/24 by Primary Key");
-printf ("Get single route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n",
-        IP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);
-
-// Get one row where nexthop=192.168.1.254
-route.nexthop   = IP4ADDR(192,168,1,254);
-ret = cross_dbGetOneRow (hRtTbl, "nexthop", &route, &route, 0);
-CHECK (ret, "Failed to get one route where nexthop=192.168.1.254");
-printf ("Get one route where nexthop=192.168.1.254: "
-        "%d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n",
-        IP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
-11
-12
-13
// Update single route 192.168.1.0/24 by Primary Key: set flags 0->1 metric 1->3
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.metric    = 3;
-route.flags     = 1;
-ret = cross_dbUpdRowByPK (hRtTbl, &route, "flags,metric", &route, 0); 
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-
-// Update routes where nexthop=192.168.1.254: set flags 0->3
-route.nexthop   = IP4ADDR(192,168,1,254);
-route.flags     = 3;
-count = cross_dbUpdateRows (hRtTbl, "nexthop", &route, "flags", &route, 0);
-printf ("Update %d routes where nexthop=10.1.2.254\n", count);
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
// Use cursor to get routes where nexthop=192.168.1.254
-cross_cursor_h hCursor;
-route.nexthop   = IP4ADDR(192,168,1,254);
-count = cross_dbQueryRows (hRtTbl, &hCursor, "nexthop", &route, 0);
-printf ("Query %d routes where nexthop=192.168.1.254\n", count);
-while (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {
-    printf ("  route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\n",
-            IP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);
-}
-cross_cursorClose (hCursor, 0);
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
// Delete single route 192.168.1.0/24 by Primary Key
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-ret = cross_dbDelRowByPK (hRtTbl, &route, 0); 
-CHECK (ret, "Failed to delete route 192.168.1.0/24 by Primary Key");
-
-// Delete routes where nexthop=192.168.1.254
-route.nexthop   = IP4ADDR(192,168,1,254);
-count = cross_dbDeleteRows (hRtTbl, "nexthop", &route, 0);
-printf ("Delete %d routes where nexthop=192.168.1.254\n", count);
-
 
 
 
 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
-10
ret = cross_dbTransBegin (hDb, 0);
-CHECK (ret, "Failed to begin transaction");
-// Update single route 192.168.1.0/24 by Primary Key: set flags 0->5
-route.prefix    = IP4ADDR(192,168,1,0);
-route.mask      = 24;   
-route.flags     = 5;
-ret = cross_dbUpdRowByPK (hRtTbl, &route, "flags", &route, 0); 
-CHECK (ret, "Failed to update route 192.168.1.0/24 by Primary Key");
-ret = cross_dbTransCommit (hDb, 0);
-CHECK (ret, "Failed to commit transaction");
-
 
 
 
 
Want to Lean More?
 
 Get Started 🧭 Tutorial 📜 
 
 
 
  
  
 
 
 
      
\ No newline at end of file
+14
xdb_res_t   *pRes;
+xdb_row_t   *pRow;
+xdb_conn_t  *pConn = xdb_open (":memory:");
+pRes = xdb_exec (pConn, "CREATE TABLE student (id INT, name CHAR(16), age INT, class CHAR(16), score INT)");
+pRes = xdb_exec (pConn, "INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)");
+pRes = xdb_exec (pConn, "SELECT * from student");
+while (NULL != (pRow = xdb_fetch_row (pRes))) {
+    xdb_print_row (pRes->row_meta, pRow);
+    printf ("\n");
+}
+xdb_free_result (pRes);
+pRes = xdb_exec (pConn, "UPDATE student set age=9 WHERE id = 2");
+pRes = xdb_exec (pConn, "DELETE FROM student WHERE id = 3");
+xdb_close (pConn);
+
1
+2
+3
+4
+5
+6
CREATE TABLE student (id INT, name CHAR(16), age INT, class CHAR(16), score INT);
+INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95);
+SELECT * FROM student;
+SELECT * FROM student WHERE id = 1;
+UPDATE student set age=9 WHERE id = 2;
+DELETE FROM student WHERE id = 3;
+


Note

This project was redesigned and rewritten from scratch. It's still in early development stage, so please DO NOT use in your project now.

\ No newline at end of file diff --git a/intro/index.html b/intro/index.html new file mode 100755 index 0000000..55b236f --- /dev/null +++ b/intro/index.html @@ -0,0 +1,51 @@ + Introduction - CrossDB

Introduction

CrossDB is a super high-performance embedded and server RDBMS. It's developed for high performance scenarios with main memory can hold whole DB.

Features

  • Support Multiple OS Platforms: Linux/Windows/MacOS/FreeBSD etc
  • Support Multiple CPU ARCH: X86/ARM/PPC/MIPS etc
  • Support OnDisk/In-memory/RamDisk/Hybrid Storage
  • Support Standard RDBMS model
  • Support MySQL like SQL
  • Support Multiple databases
  • Support Primary Key and multiple Secondary Indexes
  • Support HASH and RBTREE(TBD) Index
  • Support Multi-columns Index
  • Support Exact Match, Leftmost Match(TBD), Range Match(TBD)
  • Support Standard ACID Transaction (begin/commit/rollback)
  • Support WAL for OnDisk storage(TBD)
  • Support Multiple Threads and Multiple Processes Access
  • Support Table level read-write lock
  • Support Reader-Writer MVCC
  • Support Embedded CrossDB Shell
  • Support Multi-Statments APIs
  • Support Prepared Statments APIs
  • Super High Performance
  • Very Simple: Simple header and library file
  • Zero Config: no complex config, real out-of-the-box

Use Cases

  • You can use CrossDB In-Memory DB to manage Process Runtime Data to replace STL or hand-wrting data structures.
  • You can use CrossDB RamDisk DB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CrossDB OnDisk DB to store data on Disk/Flash/SDD.
  • You can use CrossDB to work as a super fast cache DB.

Comments

\ No newline at end of file diff --git a/products/CHANGELOG/index.html b/products/CHANGELOG/index.html deleted file mode 100644 index de9d4a6..0000000 --- a/products/CHANGELOG/index.html +++ /dev/null @@ -1 +0,0 @@ - Change Log - CrossDB

Change Log

0.5.0 (2023-06-26)

Features

  • CrossDB command line tool crossdb-cli is released
  • Optimize insert/update/query/delete performance
  • Add new API cross_matchCreate and cross_matchFree
  • DML APIs supports cross_fields_h and cross_match_h

Bug Fixes

0.4.0 (2023-06-20)

Features

  • Support FreeBSD(X64)
  • Optimize insert/update/query/delete performance
  • Add new API cross_fieldsCreate and cross_fieldsFree

Bug Fixes

0.3.0 (2023-06-13)

Features

  • Support MacOS (X64 and ARM64)
  • Change CROSS_DB_XXX to CROSS_XXX

Bug Fixes

  • cross_dbTblCreate flags CROSS_DB_RBTREE doesn't create Primary Key Index type correctly

0.2.0 (2023-06-07)

Features

  • Support Windows
  • Support Linux ARM64

Bug Fixes

0.1.0 (2023-06-03)

  • Initial release

Features

Bug Fixes

\ No newline at end of file diff --git a/products/download/index.html b/products/download/index.html deleted file mode 100644 index fdb24f3..0000000 --- a/products/download/index.html +++ /dev/null @@ -1 +0,0 @@ - Download - CrossDB

Download

We provide precompiled binaries for CrossDB components.
Download the latest release of CrossDB for your platform.

CrossDB v0.5.0

File Name OS Arch
crossdb-0.5.0-linux-x64.tar.gz Linux X64
crossdb-0.5.0-linux-arm64.tar.gz Linux ARM64
crossdb-0.5.0-windows-x64.zip Windows X64
crossdb-0.5.0-windows-x86.zip Windows X86
crossdb-0.5.0-macos.tar.gz MacOS X64 and ARM64
crossdb-0.5.0-freebsd-x64.tar.gz FreeBSD X64

CrossDB v0.4.0

File Name OS Arch
crossdb-0.4.0-linux-x64.tar.gz Linux X64
crossdb-0.4.0-linux-arm64.tar.gz Linux ARM64
crossdb-0.4.0-windows-x64.zip Windows X64
crossdb-0.4.0-macos.tar.gz MacOS X64 and ARM64
crossdb-0.4.0-freebsd-x64.tar.gz FreeBSD X64

CrossDB v0.3.0

File Name OS Arch
crossdb-0.3.0-linux-x64.tar.gz Linux X64
crossdb-0.3.0-linux-arm64.tar.gz Linux ARM64
crossdb-0.3.0-windows-x64.zip Windows X64
crossdb-0.3.0-macos.tar.gz MacOS X64 and ARM64

CrossDB v0.2.0

File Name OS Arch
crossdb-0.2.0-linux-x64.tar.gz Linux X64
crossdb-0.2.0-linux-arm64.tar.gz Linux ARM64
crossdb-0.2.0-windows-x64.zip Windows X64

CrossDB v0.1.0

File Name OS Arch
crossdb-0.1.0-linux-x64.tar.gz Linux X64
\ No newline at end of file diff --git a/products/price/index.html b/products/price/index.html deleted file mode 100644 index 62bcc84..0000000 --- a/products/price/index.html +++ /dev/null @@ -1,18 +0,0 @@ - Price and Buy - CrossDB

Price and Buy

Item Free Bronze Silver Gold Platinum Diamond
DB/Process 2 16 32 64 128 128
Table/DB 16 128 256 512 1024 1024
Max Records/Table 200K 2M 10M 50M 200M 1B
Max Row Size 1K 8K 16K 32K 64K 64K
Max Columns/Table 256 1024 2048 3072 4096 4096
Max Index/Table 4 8 12 12 16 16
Max Trigger/Table 4 8 12 12 16 16
Max Foreign key/Table 4 8 12 12 16 16
CPU Core 1 2 4 8 16 512
Price $0 $100 $250 $500 $1000 $2000
Buy Free Buy now! Buy now! Buy now! Buy now! Buy now!
Buy Free BuyNow BuyNow BuyNow BuyNow BuyNow
\ No newline at end of file diff --git a/products/products/index.html b/products/products/index.html deleted file mode 100644 index 0a8c68f..0000000 --- a/products/products/index.html +++ /dev/null @@ -1 +0,0 @@ - Products - CrossDB

Products

CrossDB Embedded Database Core

CrossDB is a powerful High Performance Embedded Database which follows traditional RDBMS Model. It's developed to improve development efficiency for embedded programmers.

CrossDB combines On-Disk In-Memory and On-RamDisk data storage in a single embedded database system, so developers can optimize applications for speed and persistence, giving you flexibility to prioritize between performance, cost, power, and space-conserving storage options.

Get Started 🧭 Read More 📜

\ No newline at end of file diff --git a/products/sponsor/index.html b/products/sponsor/index.html deleted file mode 100644 index bcc996a..0000000 --- a/products/sponsor/index.html +++ /dev/null @@ -1 +0,0 @@ - Sponsor - CrossDB

Sponsor

\ No newline at end of file diff --git a/replication/logic-replication/index.html b/replication/logic-replication/index.html new file mode 100755 index 0000000..b89eb9c --- /dev/null +++ b/replication/logic-replication/index.html @@ -0,0 +1,51 @@ + Logical Replication - CrossDB

Logical Replication

Create Replication

CREATE REPLICATION rep_name [replication_option] ...
+
+replication_option:
+    HOST = xx 
+  | PORT = xx 
+  | USER = xx 
+  | PASSWORD = xx 
+  | DB = (dbname,...)
+  | INITSYNC = {true | false}
+  | AUDIT = {true | false}
+  | SCHEMA = {true | false}
+  | ENABLE = {true | false}
+

Change Replication

ALTER REPLICATION rep_name [replication_option] ...
+
+replication_option:
+    ENABLE = {true | false}
+  | USER = xx 
+  | PASSWORD = xx 
+

Drop Replication

DROP REPLICATION [IF EXISTS] rep_name
+

Show Replication

SHOW REPLICATION
+

Comments

\ No newline at end of file diff --git a/replication/pubsub/index.html b/replication/pubsub/index.html new file mode 100755 index 0000000..5906bdf --- /dev/null +++ b/replication/pubsub/index.html @@ -0,0 +1,56 @@ + PUBSUB - CrossDB

PUBSUB

Create Subscription

CREATE SUBSCRIPTION sub_name [subscripton_option] ...
+
+subscripton_option:
+    HOST = xx 
+  | PORT = xx 
+  | USER = xx 
+  | PASSWORD = xx 
+  | SRCDB = db_name
+  | DB = db_name
+  | INITSYNC = {FULL | SCHEMA | DATA | NONE}
+  | OFFSET = 0, -1
+  | AUDIT = {true | false}
+  | ENABLE = {true | false}
+
+    FILTERS (select * from xxx where xx into mytable)
+
+  | TABLES = (tbl_name,...)
+

Change Replication

ALTER REPLICATION rep_name [replication_option] ...
+
+replication_option:
+    ENABLE = {true | false}
+  | USER = xx 
+  | PASSWORD = xx 
+

Drop Replication

DROP REPLICATION [IF EXISTS] rep_name
+

Show Replication

SHOW REPLICATION
+

Comments

\ No newline at end of file diff --git a/replication/replication/index.html b/replication/replication/index.html new file mode 100755 index 0000000..43e0068 --- /dev/null +++ b/replication/replication/index.html @@ -0,0 +1,66 @@ + Replication - CrossDB

Replication

Create Replication

CREATE REPLICATION rep_name [replication_option] ...
+
+replication_option:
+    HOST = xx 
+  | PORT = xx 
+  | USER = xx 
+  | PASSWORD = xx 
+  | DB = db_name
+  | TABLES = (tbl_name,...)
+  | TODB = db_name
+  | INITSYNC = {FULL | SCHEMA | DATA | NONE}
+  | AUDIT = {true | false}
+  | ENABLE = {true | false}
+
+    FILTERS (select * from xxx where xx into mytable)
+

Change Replication

ALTER REPLICATION rep_name [replication_option] ...
+
+replication_option:
+    ENABLE = {true | false}
+  | OFFSET = 0, -1
+  | USER = xx 
+  | PASSWORD = xx 
+

Drop Replication

DROP REPLICATION [IF EXISTS] rep_name
+

Show Replication

SHOW REPLICSTATUS
+    show secondary status
+
+SHOW REPLICATION
+    show my replica status
+
+role: master/slave
+
+SHOW MASTER STATUS
+
+SHOW REPLICA STATUS [FOR CHANNEL channel]
+SHOW SLAVE STATUS [FOR CHANNEL channel]
+

Comments

\ No newline at end of file diff --git a/search/search_index.json b/search/search_index.json old mode 100644 new mode 100755 index af12611..cd5cbb8 --- a/search/search_index.json +++ b/search/search_index.json @@ -1 +1 @@ -{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"

CrossDBThe Fastest Embedded Database in the world \u2728

Quick Learn\ud83e\udded Benchmark \ud83d\udcdc Download \ud83d\udcbe

\ud83c\udf0c RDBMS Model

Follows standard RDBMS model: DB, Table, Column, Row, Index, Cursor, Transaction, etc.

\u26fd Storage Mode

Supports On-Disk database, In-Memory database and optimized On-RamDisk database.

\ud83d\udd31 Transaction

Support the ACID (Atomic, Consistent, Isolated and Durable) principles.

\ud83d\ude8a C Struct ORM

Supports using C Struct to access DB directly. Almost all C data types are supported.

\ud83d\ude80 Native APIs

Well desgined high Performance Native APIs to define and manipulate DB.

\ud83d\udcae Auto-Upgrade

Change your struct, existing table will be upgraded automatically for you.

\u267b\ufe0f ISSU

Put all your program ISSU data into CrossDB, then you get In-Service Software Upgrade feature easily.

\ud83c\udf04 Out-of-the-Box

No complex configuration or provisioning, single header file and library. Very easy to learn and use.

"},{"location":"#crossdb-model","title":"CrossDB Model","text":""},{"location":"#crossdb-cli","title":"CrossDB CLI","text":""},{"location":"#lean-in-5m","title":"Learn CrossDB in 5 Minutes","text":"\ud83d\udef6 Schema\u2699\ufe0f Create DB\u269c\ufe0f Insert Rows\ud83d\ude80 Query Rows\ud83d\udd2b Update Rows\ud83c\udfa1 Cursor Query\u2702\ufe0f Delete Rows\ud83c\udf04 Transaction 
typedef struct {\nuint32_t            prefix;\nuint8_t             mask;\nuint32_t            nexthop;\nuint8_t             metric;\nchar                intf[16];\nuint32_t            birth;\nuint32_t            flags;\n} route_t;\n#undef  CROSS_STRUCT_NAME\n#define CROSS_STRUCT_NAME   route_t\ncross_field_t   route_schema[] = {\nCROSS_FIELD (prefix,    UINT,   IPv4, 0),\nCROSS_FIELD (mask,      UINT,   DFT,  0),\nCROSS_FIELD (nexthop,   UINT,   IPv4, 0),\nCROSS_FIELD (metric,    UINT,   DFT,  0),\nCROSS_FIELD (intf,      CHAR,   DFT,  0),\nCROSS_FIELD (birth,     UINT,   TS,   0),\nCROSS_FIELD (flags,     UINT,   HEX,  0),\nCROSS_END (route_t)\n};\n
#define CHECK(ret,str)      if (ret < 0) {  printf (str\": %s\\n\", cross_errMsg(ret)); return -1; }\ncross_db_h      hDb;\ncross_tbl_h     hRtTbl;\ncross_ret       ret;\nroute_t         route;  cross_rowid     count;\n// Create database\nret = cross_dbCreate (&hDb, \"db_data/example\", 0);\nCHECK (ret, \"Failed to create db: example\");\n// Create table: route (Primary Key: prefix,mask)\nret = cross_dbTblCreate (hDb, &hRtTbl, \"route\", route_schema, \"prefix,mask\", 0);\nCHECK (ret, \"Failed to create table: route table\");\n// Create index on nexthop: idx_nexthop\nret = cross_dbIdxCreate (hRtTbl, \"idx_nexthop\", \"nexthop\", 0);\nCHECK (ret, \"Failed to create index: idx_nexthop\");\n
#define IP4ADDR(a,b,c,d)    ((a)<<24|(b)<<16|(c)<<8|(d))\n// Insert route 192.168.1.0/24->192.168.1.254\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(192,168,1,254);\nroute.metric    = 1;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth1\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert route 192.168.1.0/24\");\n
#define IP4STR(ip)              ip>>24,(ip>>16)&0xff,(ip>>8)&0xff,ip&0xff\n// Get single route 192.168.1.0/24 by Primary Key\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0); CHECK (ret, \"Failed to get route 192.168.1.0/24 by Primary Key\");\nprintf (\"Get single route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\",\nIP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);\n// Get one row where nexthop=192.168.1.254\nroute.nexthop   = IP4ADDR(192,168,1,254);\nret = cross_dbGetOneRow (hRtTbl, \"nexthop\", &route, &route, 0);\nCHECK (ret, \"Failed to get one route where nexthop=192.168.1.254\");\nprintf (\"Get one route where nexthop=192.168.1.254: \"\n\"%d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\",\nIP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);\n
// Update single route 192.168.1.0/24 by Primary Key: set flags 0->1 metric 1->3\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.metric    = 3;\nroute.flags     = 1;\nret = cross_dbUpdRowByPK (hRtTbl, &route, \"flags,metric\", &route, 0); CHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\n// Update routes where nexthop=192.168.1.254: set flags 0->3\nroute.nexthop   = IP4ADDR(192,168,1,254);\nroute.flags     = 3;\ncount = cross_dbUpdateRows (hRtTbl, \"nexthop\", &route, \"flags\", &route, 0);\nprintf (\"Update %d routes where nexthop=10.1.2.254\\n\", count);\n
// Use cursor to get routes where nexthop=192.168.1.254\ncross_cursor_h hCursor;\nroute.nexthop   = IP4ADDR(192,168,1,254);\ncount = cross_dbQueryRows (hRtTbl, &hCursor, \"nexthop\", &route, 0);\nprintf (\"Query %d routes where nexthop=192.168.1.254\\n\", count);\nwhile (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {\nprintf (\"  route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\",\nIP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);\n}\ncross_cursorClose (hCursor, 0);\n
// Delete single route 192.168.1.0/24 by Primary Key\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbDelRowByPK (hRtTbl, &route, 0); CHECK (ret, \"Failed to delete route 192.168.1.0/24 by Primary Key\");\n// Delete routes where nexthop=192.168.1.254\nroute.nexthop   = IP4ADDR(192,168,1,254);\ncount = cross_dbDeleteRows (hRtTbl, \"nexthop\", &route, 0);\nprintf (\"Delete %d routes where nexthop=192.168.1.254\\n\", count);\n
ret = cross_dbTransBegin (hDb, 0);\nCHECK (ret, \"Failed to begin transaction\");\n// Update single route 192.168.1.0/24 by Primary Key: set flags 0->5\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 5;\nret = cross_dbUpdRowByPK (hRtTbl, &route, \"flags\", &route, 0); CHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\nret = cross_dbTransCommit (hDb, 0);\nCHECK (ret, \"Failed to commit transaction\");\n
"},{"location":"#want-to-lean-more","title":"Want to Lean More?","text":"

Get Started \ud83e\udded Tutorial \ud83d\udcdc

"},{"location":"about/about/","title":"About CrossDB","text":"

CrossDB was developed from 2019. It's designed to be a high-performance embedded database for application, OLTP or embedded system.

To improve development efficiency, more and more features are designed and developed, and it's really an amazing tools for developers.

Our mission is to deliver fantastic tools for embedded database application, high-frequency OLTP and embedded system development with a high-performance embedded database, database-driven development framework and distribution development solution.

CrossDB is very easy to use, high efficiency, powerful and fast enough to help you refactor your project, reduce proudcut time to market and make your project successful.

"},{"location":"about/about/#contact-us","title":"Contact Us","text":"

support@crossdb.org

"},{"location":"about/buy/","title":"How to Buy","text":""},{"location":"about/buy/#buy-crossdb-windows-x64-for-server","title":"Buy CrossDB Windows x64 for Server","text":"

Primary Advanced Professional Enterprise

"},{"location":"about/buy/#buy-crossdb-linux-x64-for-server","title":"Buy CrossDB Linux x64 for Server","text":"

Primary Advanced Professional Enterprise

"},{"location":"about/buy/#buy-crossdb-linux-x64-for-embedded-system1000devices","title":"Buy CrossDB Linux x64 for Embedded System(1000devices)","text":"

Primary Advanced Professional Enterprise

"},{"location":"about/buy/#buy-crossdb-linux-arm64-for-embedded-system-1000devices","title":"Buy CrossDB Linux arm64 for Embedded System (1000devices)","text":"

Primary Advanced Professional Enterprise

"},{"location":"about/buy/#crossdb-windows-x64-professional-version-from-paddle","title":"CrossDB Windows x64 Professional Version from Paddle","text":"

Buy Now! Subscribe Now!

Buy Bronze Version now! Buy Silver Version now! Buy Gold Version now! Buy Gold Version now! Buy Platinum Version now! Buy Diamond Version now!

Enterprise

We also have Volume Discounts when you buy. If you need more licenses than shown here, please contact us for a quote.

"},{"location":"about/community/","title":"Community","text":""},{"location":"about/community/#report-bugs","title":"Report Bugs","text":"

Github Issues

"},{"location":"about/community/#require-fetures","title":"Require Fetures","text":"

Github Fetures

"},{"location":"about/community/#discussion","title":"Discussion","text":"

Github Discussion

"},{"location":"about/faq/","title":"FAQ","text":""},{"location":"about/faq/#why-crossdb","title":"Why CrossDB","text":""},{"location":"about/privacy/","title":"Privacy Notice","text":""},{"location":"about/privacy/#introduction","title":"Introduction","text":"

CrossDB Software has designed its website so that, in general, you can browse the CrossDB website without being asked for any personal information. Exceptions to this are when you specifically ask for information, request an evaluation copy of our software or download CrossDB trial software.

This policy covers how CrossDB Software treats personal information that CrossDB may collect and receive via the CrossDB website. Your use of the CrossDB website constitutes acceptance of and agreement with the CrossDB Software Privacy Policy. CrossDB reserves the right to change the CrossDB Software Privacy Policy at any time without notice.

"},{"location":"about/privacy/#information-collection-and-use","title":"Information collection and use","text":"

CrossDB collects personal information when you request for information, request an evaluation of our software or when you download trial software. This information includes information about you such as name, business name and address, telephone and fax numbers, email address, operating system, and hardware environment. Providing information is voluntary. You may provide this information by filling out an on-line form, by email or other communication to CrossDB. However, you may not receive the product information, software evaluation or downloads if you choose not to provide the requested information.

CrossDB Software's use of the information you provided will be to send you the information that you requested. CrossDB Software may also use the information to provide you with additional information about our products and services or market research. The information may be used to follow up with you regarding information supplied, software you downloaded or evaluated.

CrossDB does not sell information collected from users of CrossDB website to other organizations. CrossDB may share the information collected with its subsidiaries and its affiliates such as partners and distributors.

"},{"location":"about/privacy/#general-information","title":"General Information","text":"

CrossDB web servers record standard information about CrossDB Web accesses such as IP addresses, your server name, browser type, etc. when you visit our website. CrossDB uses this for information about usage at CrossDB Software's website. The information submitted to CrossDB Software via the Web may be processed, stored and used outside the country where it was submitted.

Links to third party websites are provided for your convenience. By using these links, you will no longer be on the CrossDB Software website and the CrossDB Software Privacy Policy will no longer apply.

"},{"location":"about/privacy/#contact-us","title":"Contact us","text":"

If you have any question about the information you provided to us or about the CrossDB Software Privacy Policy, please contact us at wang_junchuan@163.com

"},{"location":"about/refund/","title":"Refund Policy","text":"

Thank you for using CrossDB Embedded Database! We offer refund and/or exchange within the first 30 days of your purchase, if 30 days have passed since your purchase, you will not be offered a refund and/or exchange of any kind.

"},{"location":"about/terms/","title":"Terms of Service","text":"

CrossDB provides this Website for your information and use, subject to the terms set forth below. By accessing this Website, you agree to accept the following terms.

"},{"location":"about/terms/#general-terms","title":"General terms","text":"

In no event will CrossDB be liable for any damages including, without limitation, indirect, incidental, consequential, special or exemplary damages, that arise out of or relate to the use of or inability to use the CrossDB Website, even if CrossDB has been advised of the possibility of such damages.

Any and all information provided on CrossDB\u2019s Website is provided \u201cas is,\u201d with no warranty as to accuracy or content.

CrossDB does not adopt or endorse the views of any third party Websites linked to or from the CrossDB Website.

CrossDB reserves the right to change, modify, add or remove any of these terms at any time.

"},{"location":"backup/","title":"Index","text":"

CrossDBHigh Performance Embedded Database Development Framework\u2728

Start \ud83e\udded Docs \ud83d\udca1 Download \ud83d\udca1

\ud83c\udf08 RDBMS

CDB follows standard RDBMS model DB, Tabel, Record, Index, Cursor, Transaction, SQL.

\ud83d\ude8a C ORM

CDB supports C Struct to access DB direclty. Almost all C data types are supported Enum, Union, Bitfield, Array, nested struct, etc.

\ud83d\ude80 Trigger

CDB Supports Foreign Key, Trigger and Cascade Trigger. They can be used to do Data-driven programming and unit test.

\u26fd PubSub

CDB supports PubSbub to subscribe DB from other source.

\ud83d\udcae RPC

CDB supports C struct serailization and a gRPC like RPC framework for C language in future.

\ud83d\udd31 Connector

CDB supports C/C++ and Python Connectors now, will support more connectors later.

\u267b\ufe0f Best Practice

Use CDB to Manage Data, do Data-Driven Programming, ISSU, distribut provisioning, etc.

\ud83c\udf04 Troubleshooting

CDB supports CLI, WEB, Staticstics, Status, Log, DB backup and restore to do troubleshooting.

"},{"location":"backup/#5-minutes-to-learn-crossdb","title":"5 Minutes to Learn CrossDB","text":"\ud83c\udf04 Define Schema\u267b\ufe0f Create Database Table and Index\ud83d\udcae Insert Rows\ud83c\udf04 Query Rows\ud83d\udcae Update Rows\ud83d\udcae Delete Rows 
typedef struct {\nuint32_t            prefix;\nuint8_t             mask;\nuint32_t            nexthop;\nuint8_t             metric;\nchar                intf[16];\nuint32_t            birth;\nuint32_t            flags;\n} route_t;\n#undef  CROSS_STRUCT_NAME\n#define CROSS_STRUCT_NAME   route_t\ncross_field_t   route_schema[] = {\nCROSS_FIELD (prefix,    UINT,   IPv4, 0),\nCROSS_FIELD (mask,      UINT,   DFT,  0),\nCROSS_FIELD (nexthop,   UINT,   IPv4, 0),\nCROSS_FIELD (metric,    UINT,   DFT,  0),\nCROSS_FIELD (type,      UINT,   DFT,  0),\nCROSS_FIELD (intf,      CHAR,   DFT,  0),\nCROSS_FIELD (birth,     UINT,   TS,   0),\nCROSS_FIELD (flags,     UINT,   HEX,  0),\nCROSS_END (route_t)\n};\n
// Open ondisk database (create if not exist)\nprintf (\"Open databse: mydb\\n\");\nret = cross_dbOpen (&hDb, \"mydb\", CROSS_CREATE|CROSS_ONDISK);\nCHECK (ret, \"Failed to open mydb\");\n// Open table (create if not exist), PrimaryKey is ipAddr,mask\nprintf (\"Open table: route\\n\");\nret = cross_tblOpen (hDb, &hRtTbl, \"route\", route_schema, \"ipAddr,mask\", CROSS_CREATE);\nCHECK (ret, \"Failed to open route table\");\n// Create index on nexthop to accelarate lookup by nexthop\nprintf (\"Create index: idx_nexthpp[\\n\");\nret = cross_idxCreate (hRtTbl, \"idx_nexthop\", \"nexthop\", 0);\nCHECK (ret, \"Failed to create index idx_nexthop\");\n
printf (\"Insert route 192.168.1.0/24->192.168.1.254\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(10,1,2,254);\nroute.metric    = 1;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth1\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert row\");\nprintf (\"Insert route 10.1.1.0/24->10.1.2.254\\n\");\nroute.prefix    = IP4ADDR(100,1,0,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(10,1,1,254);\nroute.metric    = 1;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth2\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert row\");\nprintf (\"Insert route 10.1.2.0/24->10.1.2.254\\n\");\nroute.prefix    = IP4ADDR(10,1,1,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(10,1,2,254);\nroute.metric    = 2;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth2\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert row\");\n
// Get all rows count\ncount = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);\nprintf (\"Total %d routes\\n\", count);\n// Get rows count where nexthop=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\ncount = cross_dbGetRowsCount (hRtTbl, \"nexthop\", &route, 0);\nprintf (\"There're %d routes where nexthop=10.1.2.254\\n\", count);\n// Get single route 192.168.1.0/24 by PK\nmemset (&route, 0, sizeof(route));\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbGetRowByPk (hRtTbl, &route, &route, 0); CHECK (ret, \"Failed to get row by PK\");\nprintf (\"Get single route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\", IP4STR(route.prefix), route.mask, IP4STR(route.prefix), route.intf, route.metric, route.flags);\n// Use cursor to get routes where nexthop=10.1.2.254\ncross_cursor_h hCursor = NULL;\nroute.nexthop   = IP4ADDR(10,1,2,254);\ncount = cross_dbQueryRowsCursor (hRtTbl, &hCursor, \"nexthop\", &route, 0);\nCHECK (count, \"Failed to query rows\");\nprintf (\"Get %d routes where nexthop=10.1.2.254\\n\", count);\nwhile (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {\nprintf (\"  route: %d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\", IP4STR(route.prefix), route.mask, IP4STR(route.prefix), route.intf, route.metric, route.flags);\n}\ncross_cursorFree (hCursor);\n
printf (\"Update single route 192.168.1.0/24 by PK: set flags 0->0x10\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 0x10;\nret = cross_dbUpdRowByPk (hRtTbl, &route, \"flags\", &route, 0); CHECK (ret, \"Failed to update row by PK\");\n// Update routes where nexthop=10.1.2.254: set flags 0->0x20\nroute.nexthop   = IP4ADDR(10,1,2,254);\nroute.flags     = 0x20;\ncount = cross_dbUpdateRows (hRtTbl, \"nexthop\", &route, \"flags\", &route, 0);\nCHECK (count, \"Failed to update rows\");\nprintf (\"Update %d routes where nexthop=10.1.2.254\\n\", count);  
printf (\"Delete single route 192.168.1.0/24 by PK\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbDelRowByPk (hRtTbl, &route, 0); CHECK (ret, \"Failed to delete row by PK\");\n// Delete routes where nexthop=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\ncount = cross_dbDeleteRows (hRtTbl, \"nexthop\", &route, 0);\nCHECK (count, \"Failed to update rows\");\nprintf (\"Update %d routes where nexthop=10.1.2.254\\n\", count);\n
"},{"location":"backup/#sponsors","title":"Sponsors","text":"

Note

Some text here

Some text here

Some text here

Addtional Content

Warning

Some text here

Some text here

Some text here

Addtional Content

"},{"location":"backup/features/","title":"Key Features","text":""},{"location":"backup/features/#high-performance","title":"High Performance","text":""},{"location":"backup/features/#acid-transactions","title":"ACID Transactions","text":"

CrossDB's transactions support the ACID (Atomic, Consistent, Isolated and Durable) principles, which safeguard data integrity by guaranteeing that updates will complete together, or the database will roll back to a pre-transaction state.

"},{"location":"backup/features/#core-ramdisk-database-system-design","title":"Core RamDisk Database System Design","text":"

CrossDB was designed to be an application runtime DB from the beginning. An Ra database system eliminates disk and file I/O, cache management and other sources of latency. By working with data directly in main memory, eXtremeDB avoids the overhead of making multiple data copies and transfers inherent in disk-based DBMSs. Databases can be created in shared memory, enabling concurrent access by multiple processes. Read about the performance advantages of IMDS technology in one of our white papers.

"},{"location":"backup/features/#in-memory-ramdisk-or-on-disk-storage","title":"In-Memory RamDisk or On-Disk Storage","text":"

CrossDB supports On-Disk Storage and In-Memory In addition to the core eXtremeDB in-memory database system, McObject\u2019s eXtremeDB offers hybrid storage: on a table-by-table basis, tables can be designated for in-memory or on-disk storage (with flexible caching). Choose the best storage medium based on performance, persistence, cost and form factor.

"},{"location":"backup/features/#multi-platform","title":"Multi-Platform","text":"

eXtremeDB is available for use on all major workstation and server platforms: x32- and x64-bit versions of Windows and Linux, Linux on POWER8, Solaris Sparc, Solaris x86_64, HP-UX (Itanium), and AIX.

eXtremeDB is also available for all major embedded platforms: VxWorks, INTEGRITY, QNX, ThreadX, eCos, FreeRTOS and too many more to mention. Also x86, PowerPC, ARM, MIPS, etc.

eXtremeDB source code is available. The minimum requirement to build eXtremeDB for any platform is a 32-bit CPU and a decent quality C compiler.

"},{"location":"backup/features/#b-tree-r-tree-patricia-trie-kd-tree-trigram-and-hash-indexes","title":"B-Tree, R-Tree, Patricia Trie, KD-Tree, Trigram and Hash Indexes","text":"

eXtremeDB provides a wide range of database indexes, to boost application performance and minimize footprint. eXtremeDB offers R-trees for geospatial data, Patricia tries for IP/telecom, KD-trees for multi-dimensional data and Query-by-Example (QBE), B-trees, Trigram index for fuzzy search, hash indexes and more. For in-memory databases, rather than storing duplicate data, indexes contain only a reference to data, keeping memory requirements to an absolute minimum.

"},{"location":"backup/features/#wide-range-of-supported-data-types","title":"Wide Range of Supported Data Types","text":"

eXtremeDB supports a wide range of data types \u2014 including structures, arrays, boolean, binary, decimal, vectors and BLOBs \u2014 for maximum coding efficiency. Data can be stored in the same complex form in which it is used in the application, or as normalized relations. Learn more in our online documentation.

"},{"location":"backup/features/#single-header-and-libray","title":"\ud83c\udf08 Single header and libray","text":"

CDB follows standard RDBMS model DB, Tabel, Record, Index, Cursor, Transaction, SQL.

"},{"location":"backup/features/#c-orm","title":"\ud83d\ude8a C ORM","text":"

CDB supports C Struct to access DB direclty. Almost all C data types are supported Enum, Union, Bitfield, Array, nested struct, etc.

"},{"location":"backup/features/#trigger","title":"\ud83d\ude80 Trigger","text":"

CDB Supports Foreign Key, Trigger and Cascade Trigger. They can be used to do Data-driven programming and unit test.

"},{"location":"backup/features/#pubsub","title":"\u26fd PubSub","text":"

CDB supports PubSbub to subscribe DB from other source.

"},{"location":"backup/features/#rpc","title":"\ud83d\udcae RPC","text":"

CDB supports C struct serailization and a gRPC like RPC framework for C language in future.

"},{"location":"backup/features/#connector","title":"\ud83d\udd31 Connector","text":"

CDB supports C/C++ and Python Connectors now, will support more connectors later.

"},{"location":"backup/features/#best-practice","title":"\u267b\ufe0f Best Practice","text":"

Use CDB to Manage Data, do Data-Driven Programming, ISSU, distribut provisioning, etc.

"},{"location":"backup/features/#troubleshooting","title":"\ud83c\udf04 Troubleshooting","text":"

CDB supports CLI, WEB, Staticstics, Status, Log, DB backup and restore to do troubleshooting.

"},{"location":"backup/introduction/","title":"Introduction","text":"

CrossDB(CDB) is a powerful high-performance distrubuted embedded SQL RDBMS database. It's developed to improve development efficiency for embedded programmers.

  • You can use CDB RDBMS to Manage Program Data efficiently.
  • You can use CDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CDB RDBMS to refactor your code conveniently.
  • You can use CDB Index to optimize performance without changing your code.
  • You can use CDB Trigger to implement Data-Driven programming paradigm.
  • You can use CDB PUBSUB to subscribe DB from other process's DB either on same host or remote host.
  • You can use CDB to implement Centralize-DB programming paradigm.
  • You can use CDB eventloop to implement event-driven programming paradigm.
  • You can use CDB RPC to build distributed service.
  • You can use CDB CLI tool to debug running program's data in off-line way.
  • You can use CDB SQL to view/create/update/delete/filter program data.
  • You can use CDB Browser to view program data.
  • You can use CDB to do DB backup restore it.
  • You can use CDB DB Change Log to view DB change history with filter, backtrace, rate-limit, expiring, etc.
  • You can use Python connector to write unit test with SQL to test program.
  • You can copy the DB folders from device and open on PC/Server with cdb-cli or Python directly.
  • Use CDB DB-Driven Mode to Build Program Logic (Table Trigger, FK, auto delete, Cascade Trigger)
  • Use CDB Python Connector to do DB-Driven Unit Test
  • Use CDB Pub/Sub to Build Distributed System (Eventloop/Timer/WorkQueue, vs IPC, OOP [priv data])
  • Use CDB Pub/Sub to Build Centralized DB-Driven System (3rd lang, DM no checkpoint)
  • CDB Serialization and RPC
  • CDB SQL Connectors (RESP3, Python)
  • CDB SQL Drivers (C, Python)
"},{"location":"backup/introduction/#architecture","title":"Architecture","text":"

png relative

"},{"location":"backup/introduction/#feature-list","title":"Feature List","text":"

TDB

"},{"location":"backup/introduction/#support-os","title":"Support OS","text":"
  • Linux
  • Windows
"},{"location":"backup/introduction/#support-platform","title":"Support Platform","text":"

Following platform are tested

  • x86
  • PowerPC
  • ARM
"},{"location":"backup/introduction/#porting","title":"Porting","text":"

Log

Heart beat

"},{"location":"backup/introduction/#compiling","title":"Compiling","text":""},{"location":"backup/introduction/#linux","title":"Linux","text":"

Build library

Build example

"},{"location":"backup/introduction/#windows","title":"Windows","text":"

You can build with MinGW or MSYS2 or Cygwin

Build library MinGW MSYS2 Cygwin

Build example

"},{"location":"backup/usecases/","title":"Products","text":"

CrossDB(CDB) is a powerful high-performance distrubuted embedded SQL RDBMS database. It's developed to improve development efficiency for embedded programmers.

CrossDB combines on-disk and in-memory data storage in a single embedded database system, so developers can optimize applications for speed and persistence, giving you flexibility to prioritize between performance, cost, power, and space-conserving storage options. Offering native and SQL APIs for C, C++, and Python.

"},{"location":"backup/usecases/#feature-list","title":"Feature List","text":"
  • You can use CDB RDBMS to Manage Program Data efficiently.
  • You can use CDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CDB RDBMS to refactor your code conveniently.
  • You can use CDB Index to optimize performance without changing your code.
  • You can use CDB Trigger to implement Data-Driven programming paradigm.
  • You can use CDB PUBSUB to subscribe DB from other process's DB either on same host or remote host.
  • You can use CDB to implement Centralize-DB programming paradigm.
  • You can use CDB eventloop to implement event-driven programming paradigm.
  • You can use CDB RPC to build distributed service.
  • You can use CDB CLI tool to debug running program's data in off-line way.
  • You can use CDB SQL to view/create/update/delete/filter program data.
  • You can use CDB Browser to view program data.
  • You can use CDB to do DB backup restore it.
  • You can use CDB DB Change Log to view DB change history with filter, backtrace, rate-limit, expiring, etc.
  • You can use Python connector to write unit test with SQL to test program.
  • You can copy the DB folders from device and open on PC/Server with cdb-cli or Python directly.
  • Use CDB DB-Driven Mode to Build Program Logic (Table Trigger, FK, auto delete, Cascade Trigger)
  • Use CDB Python Connector to do DB-Driven Unit Test
  • Use CDB Pub/Sub to Build Distributed System (Eventloop/Timer/WorkQueue, vs IPC, OOP [priv data])
  • Use CDB Pub/Sub to Build Centralized DB-Driven System (3rd lang, DM no checkpoint)
  • CDB Serialization and RPC
  • CDB SQL Connectors (RESP3, Python)
  • CDB SQL Drivers (C, Python)
"},{"location":"backup/blog/avx512-in-golang/","title":"How to Use AVX512 in Golang via C Compiler","text":"

AVX512 is the latest generation of SIMD instructions released by Intel, which can process 512 bits of data in one instruction cycle, equivalent to 16 single-precision floating point numbers or 8 double-precision floating point numbers. The training and inference process of recommendation models in Gorse requires a lot of vector computation, and AVX512 can theoretically bring some acceleration effect. Unfortunately, the Go compiler does not automatically generate SIMD instructions.

MinIO had open-sourced a tool to convert Intel assembly to Go assembly c2goasm. First, the vectorized functions are implemented in C, and the assembly containing the SIMD instructions is compiled by Clang. Then, since Go assembly supports AVX512, the functions implemented by SIMD can be called through Go assembly. The c2goasm solution is very effective, however, the project has not been updated for almost 4 years, and after testing it cannot handle AVX512 instructions.

To use AVX512 in Go, we have developed a toolkit for compiling C into Go assembly functions goat and implemented a library of vectorized functions with the help of goat github.com/gorse-io/gorse/base/floats. Inheriting the idea of c2goasm, goat implements further enhancements.

  1. Starting directly from the C source code to get Go assembly functions, the user does not have to compile the C source code itself.
  2. It will also generate Go function definitions based on C function definitions, so users do not need to write Go function definitions by hand.

The post will detail the technical implementation of goat ideas, welcome to read the code github.com/gorse-io/gorse/cmd/goat.

flowchart LR\n  c[C source]-->intel[Intel assembly]\n  intel-->goasm[Go assembly]\n  c-->go[Go function definition]
"},{"location":"backup/blog/avx512-in-golang/#compile-assembly-from-c","title":"Compile Assembly from C","text":"

The C implementation of the function _mm512_mul_to multiplies two floating point arrays and then saves the result in a third array. In general, the compiler automatically generates assembly that uses SIMD. Here, intrinsic is used to ensure that SIMD instructions are generated.

#include <immintrin.h>\nvoid _mm512_mul_to(float *a, float *b, float *c, int64_t n)\n{\nint epoch = n / 16;\nint remain = n % 16;\nfor (int i = 0; i < epoch; i++)\n{\n__m512 v1 = _mm512_loadu_ps(a);\n__m512 v2 = _mm512_loadu_ps(b);\n__m512 v = _mm512_mul_ps(v1, v2);\n_mm512_storeu_ps(c, v);\na += 16;\nb += 16;\nc += 16;\n}\nif (remain >= 8)\n{\n__m256 v1 = _mm256_loadu_ps(a);\n__m256 v2 = _mm256_loadu_ps(b);\n__m256 v = _mm256_mul_ps(v1, v2);\n_mm256_storeu_ps(c, v);\na += 8;\nb += 8;\nc += 8;\nremain -= 8;\n}\nfor (int i = 0; i < remain; i++)\n{\nc[i] = a[i] * b[i];\n}\n}\n

Save the code to mm512_mul_to.c and use the following command to compile the C code to assembly as well as binary.

# Generate assembly\nclang -S -c mm512_mul_to.c -o mm512_mul_to.s \\\n-O3 -mavx -mfma -mavx512f -mavx512dq \\\n-mno-red-zone -mstackrealign -mllvm -inline-threshold=1000 \\\n-fno-asynchronous-unwind-tables -fno-exceptions -fno-rtti\n\n# Generate binary\nclang -c mm512_mul_to.c -o mm512_mul_to.o \\\n-O3 -mavx -mfma -mavx512f -mavx512dq \\\n-mno-red-zone -mstackrealign -mllvm -inline-threshold=1000 \\\n-fno-asynchronous-unwind-tables -fno-exceptions -fno-rtti\n

::: details mm512_mul_to.s

 .globl _mm512_mul_to                   # -- Begin function _mm512_mul_to\n.p2align 4, 0x90\n.type _mm512_mul_to,@function\n_mm512_mul_to:                          # @_mm512_mul_to\n# %bb.0:\npushq %rbp\nmovq %rsp, %rbp\nandq $-8, %rsp\nleaq 15(%rcx), %r9\ntestq %rcx, %rcx\ncmovnsq %rcx, %r9\nshrq $4, %r9\nmovl %r9d, %eax\nshll $4, %eax\nsubl %eax, %ecx\ntestl %r9d, %r9d\njle .LBB3_6\n# %bb.1:\nleal -1(%r9), %eax\nmovl %r9d, %r8d\nandl $3, %r8d\ncmpl $3, %eax\njb .LBB3_4\n# %bb.2:\nandl $-4, %r9d\nnegl %r9d\n.p2align 4, 0x90\n.LBB3_3:                                # =>This Inner Loop Header: Depth=1\nvmovups (%rdi), %zmm0\nvmulps (%rsi), %zmm0, %zmm0\nvmovups %zmm0, (%rdx)\nvmovups 64(%rdi), %zmm0\nvmulps 64(%rsi), %zmm0, %zmm0\nvmovups %zmm0, 64(%rdx)\nvmovups 128(%rdi), %zmm0\nvmulps 128(%rsi), %zmm0, %zmm0\nvmovups %zmm0, 128(%rdx)\nvmovups 192(%rdi), %zmm0\nvmulps 192(%rsi), %zmm0, %zmm0\nvmovups %zmm0, 192(%rdx)\naddq $256, %rdi                      # imm = 0x100\naddq $256, %rsi                      # imm = 0x100\naddq $256, %rdx                      # imm = 0x100\naddl $4, %r9d\njne .LBB3_3\n.LBB3_4:\ntestl %r8d, %r8d\nje .LBB3_6\n.p2align 4, 0x90\n.LBB3_5:                                # =>This Inner Loop Header: Depth=1\nvmovups (%rdi), %zmm0\nvmulps (%rsi), %zmm0, %zmm0\nvmovups %zmm0, (%rdx)\naddq $64, %rdi\naddq $64, %rsi\naddq $64, %rdx\naddl $-1, %r8d\njne .LBB3_5\n.LBB3_6:\ncmpl $7, %ecx\njle .LBB3_8\n# %bb.7:\nvmovups (%rdi), %ymm0\nvmulps (%rsi), %ymm0, %ymm0\nvmovups %ymm0, (%rdx)\naddq $32, %rdi\naddq $32, %rsi\naddq $32, %rdx\naddl $-8, %ecx\n.LBB3_8:\ntestl %ecx, %ecx\njle .LBB3_14\n# %bb.9:\nmovl %ecx, %ecx\nleaq -1(%rcx), %rax\nmovl %ecx, %r8d\nandl $3, %r8d\ncmpq $3, %rax\njae .LBB3_15\n# %bb.10:\nxorl %eax, %eax\njmp .LBB3_11\n.LBB3_15:\nandl $-4, %ecx\nxorl %eax, %eax\n.p2align 4, 0x90\n.LBB3_16:                               # =>This Inner Loop Header: Depth=1\nvmovss (%rdi,%rax,4), %xmm0            # xmm0 = mem[0],zero,zero,zero\nvmulss (%rsi,%rax,4), %xmm0, %xmm0\nvmovss %xmm0, (%rdx,%rax,4)\nvmovss 4(%rdi,%rax,4), %xmm0           # xmm0 = mem[0],zero,zero,zero\nvmulss 4(%rsi,%rax,4), %xmm0, %xmm0\nvmovss %xmm0, 4(%rdx,%rax,4)\nvmovss 8(%rdi,%rax,4), %xmm0           # xmm0 = mem[0],zero,zero,zero\nvmulss 8(%rsi,%rax,4), %xmm0, %xmm0\nvmovss %xmm0, 8(%rdx,%rax,4)\nvmovss 12(%rdi,%rax,4), %xmm0          # xmm0 = mem[0],zero,zero,zero\nvmulss 12(%rsi,%rax,4), %xmm0, %xmm0\nvmovss %xmm0, 12(%rdx,%rax,4)\naddq $4, %rax\ncmpq %rax, %rcx\njne .LBB3_16\n.LBB3_11:\ntestq %r8, %r8\nje .LBB3_14\n# %bb.12:\nleaq (%rdx,%rax,4), %rcx\nleaq (%rsi,%rax,4), %rdx\nleaq (%rdi,%rax,4), %rax\nxorl %esi, %esi\n.p2align 4, 0x90\n.LBB3_13:                               # =>This Inner Loop Header: Depth=1\nvmovss (%rax,%rsi,4), %xmm0            # xmm0 = mem[0],zero,zero,zero\nvmulss (%rdx,%rsi,4), %xmm0, %xmm0\nvmovss %xmm0, (%rcx,%rsi,4)\naddq $1, %rsi\ncmpq %rsi, %r8\njne .LBB3_13\n.LBB3_14:\nmovq %rbp, %rsp\npopq %rbp\nvzeroupper\nretq\n.Lfunc_end3:\n.size _mm512_mul_to, .Lfunc_end3-_mm512_mul_to\n# -- End function\n

:::

In Go, it is easy to execute the compile command with exec.Command.

"},{"location":"backup/blog/avx512-in-golang/#construct-go-code-from-assembly","title":"Construct Go Code from Assembly","text":""},{"location":"backup/blog/avx512-in-golang/#convert-assembly","title":"Convert Assembly","text":"

mm512_mul_to.o is the binary of the function after it has been compiled. By using objdump you can see that each assembly has been converted to machine code.

objdump -d mm512_mul_to.o --insn-width 16\n

::: details objdump output

0000000000000460 <_mm512_mul_to>:\n 460:   55                              push   %rbp\n 461:   48 89 e5                        mov    %rsp,%rbp\n 464:   48 83 e4 f8                     and    $0xfffffffffffffff8,%rsp\n 468:   4c 8d 49 0f                     lea    0xf(%rcx),%r9\n 46c:   48 85 c9                        test   %rcx,%rcx\n 46f:   4c 0f 49 c9                     cmovns %rcx,%r9\n 473:   49 c1 e9 04                     shr    $0x4,%r9\n 477:   44 89 c8                        mov    %r9d,%eax\n 47a:   c1 e0 04                        shl    $0x4,%eax\n 47d:   29 c1                           sub    %eax,%ecx\n 47f:   45 85 c9                        test   %r9d,%r9d\n 482:   0f 8e bc 00 00 00               jle    544 <_mm512_mul_to+0xe4>\n 488:   41 8d 41 ff                     lea    -0x1(%r9),%eax\n 48c:   45 89 c8                        mov    %r9d,%r8d\n 48f:   41 83 e0 03                     and    $0x3,%r8d\n 493:   83 f8 03                        cmp    $0x3,%eax\n 496:   72 74                           jb     50c <_mm512_mul_to+0xac>\n 498:   41 83 e1 fc                     and    $0xfffffffc,%r9d\n 49c:   41 f7 d9                        neg    %r9d\n 49f:   90                              nop\n 4a0:   62 f1 7c 48 10 07               vmovups (%rdi),%zmm0\n 4a6:   62 f1 7c 48 59 06               vmulps (%rsi),%zmm0,%zmm0\n 4ac:   62 f1 7c 48 11 02               vmovups %zmm0,(%rdx)\n 4b2:   62 f1 7c 48 10 47 01            vmovups 0x40(%rdi),%zmm0\n 4b9:   62 f1 7c 48 59 46 01            vmulps 0x40(%rsi),%zmm0,%zmm0\n 4c0:   62 f1 7c 48 11 42 01            vmovups %zmm0,0x40(%rdx)\n 4c7:   62 f1 7c 48 10 47 02            vmovups 0x80(%rdi),%zmm0\n 4ce:   62 f1 7c 48 59 46 02            vmulps 0x80(%rsi),%zmm0,%zmm0\n 4d5:   62 f1 7c 48 11 42 02            vmovups %zmm0,0x80(%rdx)\n 4dc:   62 f1 7c 48 10 47 03            vmovups 0xc0(%rdi),%zmm0\n 4e3:   62 f1 7c 48 59 46 03            vmulps 0xc0(%rsi),%zmm0,%zmm0\n 4ea:   62 f1 7c 48 11 42 03            vmovups %zmm0,0xc0(%rdx)\n 4f1:   48 81 c7 00 01 00 00            add    $0x100,%rdi\n 4f8:   48 81 c6 00 01 00 00            add    $0x100,%rsi\n 4ff:   48 81 c2 00 01 00 00            add    $0x100,%rdx\n 506:   41 83 c1 04                     add    $0x4,%r9d\n 50a:   75 94                           jne    4a0 <_mm512_mul_to+0x40>\n 50c:   45 85 c0                        test   %r8d,%r8d\n 50f:   74 33                           je     544 <_mm512_mul_to+0xe4>\n 511:   66 2e 0f 1f 84 00 00 00 00 00   nopw   %cs:0x0(%rax,%rax,1)\n 51b:   0f 1f 44 00 00                  nopl   0x0(%rax,%rax,1)\n 520:   62 f1 7c 48 10 07               vmovups (%rdi),%zmm0\n 526:   62 f1 7c 48 59 06               vmulps (%rsi),%zmm0,%zmm0\n 52c:   62 f1 7c 48 11 02               vmovups %zmm0,(%rdx)\n 532:   48 83 c7 40                     add    $0x40,%rdi\n 536:   48 83 c6 40                     add    $0x40,%rsi\n 53a:   48 83 c2 40                     add    $0x40,%rdx\n 53e:   41 83 c0 ff                     add    $0xffffffff,%r8d\n 542:   75 dc                           jne    520 <_mm512_mul_to+0xc0>\n 544:   83 f9 07                        cmp    $0x7,%ecx\n 547:   7e 1b                           jle    564 <_mm512_mul_to+0x104>\n 549:   c5 fc 10 07                     vmovups (%rdi),%ymm0\n 54d:   c5 fc 59 06                     vmulps (%rsi),%ymm0,%ymm0\n 551:   c5 fc 11 02                     vmovups %ymm0,(%rdx)\n 555:   48 83 c7 20                     add    $0x20,%rdi\n 559:   48 83 c6 20                     add    $0x20,%rsi\n 55d:   48 83 c2 20                     add    $0x20,%rdx\n 561:   83 c1 f8                        add    $0xfffffff8,%ecx\n 564:   85 c9                           test   %ecx,%ecx\n 566:   0f 8e ac 00 00 00               jle    618 <_mm512_mul_to+0x1b8>\n 56c:   89 c9                           mov    %ecx,%ecx\n 56e:   48 8d 41 ff                     lea    -0x1(%rcx),%rax\n 572:   41 89 c8                        mov    %ecx,%r8d\n 575:   41 83 e0 03                     and    $0x3,%r8d\n 579:   48 83 f8 03                     cmp    $0x3,%rax\n 57d:   73 04                           jae    583 <_mm512_mul_to+0x123>\n 57f:   31 c0                           xor    %eax,%eax\n 581:   eb 5b                           jmp    5de <_mm512_mul_to+0x17e>\n 583:   83 e1 fc                        and    $0xfffffffc,%ecx\n 586:   31 c0                           xor    %eax,%eax\n 588:   0f 1f 84 00 00 00 00 00         nopl   0x0(%rax,%rax,1)\n 590:   c5 fa 10 04 87                  vmovss (%rdi,%rax,4),%xmm0\n 595:   c5 fa 59 04 86                  vmulss (%rsi,%rax,4),%xmm0,%xmm0\n 59a:   c5 fa 11 04 82                  vmovss %xmm0,(%rdx,%rax,4)\n 59f:   c5 fa 10 44 87 04               vmovss 0x4(%rdi,%rax,4),%xmm0\n 5a5:   c5 fa 59 44 86 04               vmulss 0x4(%rsi,%rax,4),%xmm0,%xmm0\n 5ab:   c5 fa 11 44 82 04               vmovss %xmm0,0x4(%rdx,%rax,4)\n 5b1:   c5 fa 10 44 87 08               vmovss 0x8(%rdi,%rax,4),%xmm0\n 5b7:   c5 fa 59 44 86 08               vmulss 0x8(%rsi,%rax,4),%xmm0,%xmm0\n 5bd:   c5 fa 11 44 82 08               vmovss %xmm0,0x8(%rdx,%rax,4)\n 5c3:   c5 fa 10 44 87 0c               vmovss 0xc(%rdi,%rax,4),%xmm0\n 5c9:   c5 fa 59 44 86 0c               vmulss 0xc(%rsi,%rax,4),%xmm0,%xmm0\n 5cf:   c5 fa 11 44 82 0c               vmovss %xmm0,0xc(%rdx,%rax,4)\n 5d5:   48 83 c0 04                     add    $0x4,%rax\n 5d9:   48 39 c1                        cmp    %rax,%rcx\n 5dc:   75 b2                           jne    590 <_mm512_mul_to+0x130>\n 5de:   4d 85 c0                        test   %r8,%r8\n 5e1:   74 35                           je     618 <_mm512_mul_to+0x1b8>\n 5e3:   48 8d 0c 82                     lea    (%rdx,%rax,4),%rcx\n 5e7:   48 8d 14 86                     lea    (%rsi,%rax,4),%rdx\n 5eb:   48 8d 04 87                     lea    (%rdi,%rax,4),%rax\n 5ef:   31 f6                           xor    %esi,%esi\n 5f1:   66 2e 0f 1f 84 00 00 00 00 00   nopw   %cs:0x0(%rax,%rax,1)\n 5fb:   0f 1f 44 00 00                  nopl   0x0(%rax,%rax,1)\n 600:   c5 fa 10 04 b0                  vmovss (%rax,%rsi,4),%xmm0\n 605:   c5 fa 59 04 b2                  vmulss (%rdx,%rsi,4),%xmm0,%xmm0\n 60a:   c5 fa 11 04 b1                  vmovss %xmm0,(%rcx,%rsi,4)\n 60f:   48 83 c6 01                     add    $0x1,%rsi\n 613:   49 39 f0                        cmp    %rsi,%r8\n 616:   75 e8                           jne    600 <_mm512_mul_to+0x1a0>\n 618:   48 89 ec                        mov    %rbp,%rsp\n 61b:   5d                              pop    %rbp\n 61c:   c5 f8 77                        vzeroupper \n 61f:   c3                              retq   \n

:::

"},{"location":"backup/blog/avx512-in-golang/#encoding-machine-code","title":"Encoding Machine Code","text":"

Go assembly provides three instructions for representing binary machine code.

  • BYTE encodes one byte of binary data
  • WORD encodes two bytes of binary data
  • LONG encodes four bytes of binary data

If the instruction machine code length is exactly a multiple of two, for example

 498: 41 83 e1 fc and $0xfffffffc,%r9d\n

can be converted to

 LONG $0xfce18341\n

But if the length is not a multiple of two, for example

 4b2: 62 f1 7c 48 10 47 01 vmovups 0x40(%rdi),%zmm0\n

It requires a combination of three instructions to represent

 LONG $0x487cf162; WORD $0x4710; BYTE $0x01 // vmovups  64(%rdi), %zmm0\n

Note that the byte order of the instruction encoding and the byte order of the objdump output are reversed.

"},{"location":"backup/blog/avx512-in-golang/#function-definition-and-parameters","title":"Function Definition and Parameters","text":"

In C assembly, if the function has no more than 6 arguments, the arguments are stored in registers and passed to the function in the following order of placement: %rdi, %rsi, %rdx, %rcx, %r8 and %r9. However, in Go assembly, function arguments are placed in memory starting from the address held in the FP register, which requires us to move the arguments in memory to the register.

The _mm512_mul_to function has four arguments, so it is necessary to move the four arguments before the function starts.

TEXT -_mm512_mul_to(SB), $0-32\nMOVQ a+0(FP), DI\nMOVQ b+8(FP), SI\nMOVQ c+16(FP), DX\nMOVQ n+24(FP), CX\n

The function definition consists of three parts: the TEXT keyword, the name starting with - and ending with (SB), and finally the parameter memory size of 32 bytes. The information about the number of arguments is not available in assembly and needs to be obtained from the C function definition (refer to Generating go function definitions).

"},{"location":"backup/blog/avx512-in-golang/#redirect-jump-instruction","title":"Redirect Jump Instruction","text":"

x86 jump instructions jump to absolute addresses and direct encoding of jump instructions does not work. Therefore, jump instructions need to be converted to jump instructions in Go assembly.

  • Converting tags: Go assembly tags cannot start with ., so you need to remove the ..
  • Conversion commands: Go assembly jump commands are in uppercase.

After the above three steps, the final Go assembly code is obtained

::: details Go assembly

TEXT -_mm512_mul_to(SB), $0-32\nMOVQ a+0(FP), DI\nMOVQ b+8(FP), SI\nMOVQ c+16(FP), DX\nMOVQ n+24(FP), CX\nBYTE $0x55 // pushq %rbp\nWORD $0x8948; BYTE $0xe5 // movq %rsp, %rbp\nLONG $0xf8e48348 // andq $-8, %rsp\nLONG $0x0f498d4c // leaq 15(%rcx), %r9\nWORD $0x8548; BYTE $0xc9 // testq %rcx, %rcx\nLONG $0xc9490f4c // cmovnsq %rcx, %r9\nLONG $0x04e9c149 // shrq $4, %r9\nWORD $0x8944; BYTE $0xc8 // movl %r9d, %eax\nWORD $0xe0c1; BYTE $0x04 // shll $4, %eax\nWORD $0xc129 // subl %eax, %ecx\nWORD $0x8545; BYTE $0xc9 // testl %r9d, %r9d\nJLE LBB3_6\nLONG $0xff418d41 // leal -1(%r9), %eax\nWORD $0x8945; BYTE $0xc8 // movl %r9d, %r8d\nLONG $0x03e08341 // andl $3, %r8d\nWORD $0xf883; BYTE $0x03 // cmpl $3, %eax\nJB LBB3_4\nLONG $0xfce18341 // andl $-4, %r9d\nWORD $0xf741; BYTE $0xd9 // negl %r9d\nLBB3_3:\nLONG $0x487cf162; WORD $0x0710 // vmovups (%rdi), %zmm0\nLONG $0x487cf162; WORD $0x0659 // vmulps (%rsi), %zmm0, %zmm0\nLONG $0x487cf162; WORD $0x0211 // vmovups %zmm0, (%rdx)\nLONG $0x487cf162; WORD $0x4710; BYTE $0x01 // vmovups 64(%rdi), %zmm0\nLONG $0x487cf162; WORD $0x4659; BYTE $0x01 // vmulps 64(%rsi), %zmm0, %zmm0\nLONG $0x487cf162; WORD $0x4211; BYTE $0x01 // vmovups %zmm0, 64(%rdx)\nLONG $0x487cf162; WORD $0x4710; BYTE $0x02 // vmovups 128(%rdi), %zmm0\nLONG $0x487cf162; WORD $0x4659; BYTE $0x02 // vmulps 128(%rsi), %zmm0, %zmm0\nLONG $0x487cf162; WORD $0x4211; BYTE $0x02 // vmovups %zmm0, 128(%rdx)\nLONG $0x487cf162; WORD $0x4710; BYTE $0x03 // vmovups 192(%rdi), %zmm0\nLONG $0x487cf162; WORD $0x4659; BYTE $0x03 // vmulps 192(%rsi), %zmm0, %zmm0\nLONG $0x487cf162; WORD $0x4211; BYTE $0x03 // vmovups %zmm0, 192(%rdx)\nLONG $0x00c78148; WORD $0x0001; BYTE $0x00 // addq $256, %rdi\nLONG $0x00c68148; WORD $0x0001; BYTE $0x00 // addq $256, %rsi\nLONG $0x00c28148; WORD $0x0001; BYTE $0x00 // addq $256, %rdx\nLONG $0x04c18341 // addl $4, %r9d\nJNE LBB3_3\nLBB3_4:\nWORD $0x8545; BYTE $0xc0 // testl %r8d, %r8d\nJE LBB3_6\nLBB3_5:\nLONG $0x487cf162; WORD $0x0710 // vmovups (%rdi), %zmm0\nLONG $0x487cf162; WORD $0x0659 // vmulps (%rsi), %zmm0, %zmm0\nLONG $0x487cf162; WORD $0x0211 // vmovups %zmm0, (%rdx)\nLONG $0x40c78348 // addq $64, %rdi\nLONG $0x40c68348 // addq $64, %rsi\nLONG $0x40c28348 // addq $64, %rdx\nLONG $0xffc08341 // addl $-1, %r8d\nJNE LBB3_5\nLBB3_6:\nWORD $0xf983; BYTE $0x07 // cmpl $7, %ecx\nJLE LBB3_8\nLONG $0x0710fcc5 // vmovups (%rdi), %ymm0\nLONG $0x0659fcc5 // vmulps (%rsi), %ymm0, %ymm0\nLONG $0x0211fcc5 // vmovups %ymm0, (%rdx)\nLONG $0x20c78348 // addq $32, %rdi\nLONG $0x20c68348 // addq $32, %rsi\nLONG $0x20c28348 // addq $32, %rdx\nWORD $0xc183; BYTE $0xf8 // addl $-8, %ecx\nLBB3_8:\nWORD $0xc985 // testl %ecx, %ecx\nJLE LBB3_14\nWORD $0xc989 // movl %ecx, %ecx\nLONG $0xff418d48 // leaq -1(%rcx), %rax\nWORD $0x8941; BYTE $0xc8 // movl %ecx, %r8d\nLONG $0x03e08341 // andl $3, %r8d\nLONG $0x03f88348 // cmpq $3, %rax\nJAE LBB3_15\nWORD $0xc031 // xorl %eax, %eax\nJMP LBB3_11\nLBB3_15:\nWORD $0xe183; BYTE $0xfc // andl $-4, %ecx\nWORD $0xc031 // xorl %eax, %eax\nLBB3_16:\nLONG $0x0410fac5; BYTE $0x87 // vmovss (%rdi,%rax,4), %xmm0\nLONG $0x0459fac5; BYTE $0x86 // vmulss (%rsi,%rax,4), %xmm0, %xmm0\nLONG $0x0411fac5; BYTE $0x82 // vmovss %xmm0, (%rdx,%rax,4)\nLONG $0x4410fac5; WORD $0x0487 // vmovss 4(%rdi,%rax,4), %xmm0\nLONG $0x4459fac5; WORD $0x0486 // vmulss 4(%rsi,%rax,4), %xmm0, %xmm0\nLONG $0x4411fac5; WORD $0x0482 // vmovss %xmm0, 4(%rdx,%rax,4)\nLONG $0x4410fac5; WORD $0x0887 // vmovss 8(%rdi,%rax,4), %xmm0\nLONG $0x4459fac5; WORD $0x0886 // vmulss 8(%rsi,%rax,4), %xmm0, %xmm0\nLONG $0x4411fac5; WORD $0x0882 // vmovss %xmm0, 8(%rdx,%rax,4)\nLONG $0x4410fac5; WORD $0x0c87 // vmovss 12(%rdi,%rax,4), %xmm0\nLONG $0x4459fac5; WORD $0x0c86 // vmulss 12(%rsi,%rax,4), %xmm0, %xmm0\nLONG $0x4411fac5; WORD $0x0c82 // vmovss %xmm0, 12(%rdx,%rax,4)\nLONG $0x04c08348 // addq $4, %rax\nWORD $0x3948; BYTE $0xc1 // cmpq %rax, %rcx\nJNE LBB3_16\nLBB3_11:\nWORD $0x854d; BYTE $0xc0 // testq %r8, %r8\nJE LBB3_14\nLONG $0x820c8d48 // leaq (%rdx,%rax,4), %rcx\nLONG $0x86148d48 // leaq (%rsi,%rax,4), %rdx\nLONG $0x87048d48 // leaq (%rdi,%rax,4), %rax\nWORD $0xf631 // xorl %esi, %esi\nLBB3_13:\nLONG $0x0410fac5; BYTE $0xb0 // vmovss (%rax,%rsi,4), %xmm0\nLONG $0x0459fac5; BYTE $0xb2 // vmulss (%rdx,%rsi,4), %xmm0, %xmm0\nLONG $0x0411fac5; BYTE $0xb1 // vmovss %xmm0, (%rcx,%rsi,4)\nLONG $0x01c68348 // addq $1, %rsi\nWORD $0x3949; BYTE $0xf0 // cmpq %rsi, %r8\nJNE LBB3_13\nLBB3_14:\nWORD $0x8948; BYTE $0xec // movq %rbp, %rsp\nBYTE $0x5d // popq %rbp\nWORD $0xf8c5; BYTE $0x77 // vzeroupper\nBYTE $0xc3 // retq\n

:::

"},{"location":"backup/blog/avx512-in-golang/#generate-go-function-definitions","title":"Generate Go Function Definitions","text":"

Converting C function definitions to Go functions requires a C parser. cznic implements a C-to-Go converter in Go cc/v3 which provides a C parser. The function definition conversion consists of two parts.

  • Function name: Directly use the function name of the C function as the Go function name
  • Function arguments: Need to check that function arguments must be of the 64-bit type, passed in Go functions using unsafe.Pointer

The definition of the _mm512_mul_to conversion to a Go function is as follows

import \"unsafe\"\n//go:noescape\nfunc _mm512_mul_to(a, b, c, n unsafe.Pointer)\n
"},{"location":"backup/blog/avx512-in-golang/#build-with-go-generate","title":"Build with go generate","text":"

For generated code, the go generate command can run custom commands to do conversions from C functions to Go functions. gorse added the go generate command to the floats_amd64.go file, which executes go generate . /... to automatically generate Go vectorized functions.

//go:generate go run ... /... /cmd/goat src/floats_avx.c -O3 -mavx\n//go:generate go run ... /... /cmd/goat src/floats_avx512.c -O3 -mavx -mfma -mavx512f -mavx512dq\n
"},{"location":"backup/blog/avx512-in-golang/#function-calls","title":"Function Calls","text":"

Calling the vectorized function is not a simple task, because only new CPUs support AVX512, and some older CPUs do not even support AVX2, which requires the following steps when calling the vectorized function.

  1. Provide a non-vectorized function implementation for CPUs that do not support AVX512.
  2. Detect in the init function whether the currently running CPU supports AVX512.
  3. Wrapping an outer function automatically selects a vectorized or non-vectorized function depending on CPU instruction support.

The complete wrapper is as follows

import (\n\"github.com/klauspost/cpuid/v2\"\n\"unsafe\"\n)\n// CPU instruction flag\nvar impl = Default\nconst (\nDefault int = iota\nAVX512\n)\nfunc init() {\n// Check if the CPU supports AVX512\nif cpuid.CPU.Supports(cpuid.AVX512F, cpuid.AVX512DQ) {\nimpl = AVX512\n}\n}\n// Non-vectorized implementation\nfunc mulTo(a, b, c []float32) {\nfor i := range a {\nc[i] = a[i] * b[i]\n}\n}\n// Wrapper functions\nfunc MulTo(a, b, c []float32) {\nif len(a) ! = len(b) || len(a) ! = len(c) {\npanic(\"floats: slice lengths do not match\")\n}\n// Automatic selection of functions for execution\nswitch impl {\ncase AVX512:\n_mm512_mul_to(unsafe.Pointer(&a[0]), unsafe.Pointer(&b[0]), unsafe.Pointer(&c[0]), unsafe.Pointer(uintptr(len(a))))\ndefault:\nmulTo(a, b, c)\n}\n}\n

::: note

Go assembly should be saved in a file with the .s extension and the Go definition and wrapper should be saved in Go files. They must be in the same directory with the same package name.

:::

"},{"location":"backup/blog/avx512-in-golang/#summary","title":"Summary","text":"

Finally, we benchmark the performance of the non-vectorized function and the vectorized function.

  • Vectorized functions take significantly less time than non-vectorized functions, especially when the vector length is 128.
  • The AVX512 implementation has a slight performance increase relative to the AVX2 implementation.

::: echarts Vectorized Implementation Performance Benchmark

{\n\"legend\": {\n\"data\": [\"SISD\", \"AVX2\", \"AVX512\"]\n},\n\"xAxis\": {\n\"name\": \"vector-length\",\n\"type\": \"category\",\n\"data\": [16, 32, 64, 128]\n},\n\"yAxis\": {\n\"name\": \"ns/op\",\n\"type\": \"value\"\n},\n\"series\": [\n{\n\"name\": \"SISD\",\n\"data\": [10.56, 23.13, 38.01, 67.05],\n\"type\": \"bar\",\n\"showBackground\": true,\n\"backgroundStyle\": {\n\"color\": \"rgba(180, 180, 180, 0.2)\"\n}\n},\n{\n\"name\": \"AVX2\",\n\"data\": [5.046, 5.138, 6.074, 8.294],\n\"type\": \"bar\",\n\"showBackground\": true,\n\"backgroundStyle\": {\n\"color\": \"rgba(180, 180, 180, 0.2)\"\n}\n},\n{\n\"name\": \"AVX512\",\n\"data\": [4.741, 5.417, 5.583, 6.425],\n\"type\": \"bar\",\n\"showBackground\": true,\n\"backgroundStyle\": {\n\"color\": \"rgba(180, 180, 180, 0.2)\"\n}\n}\n]\n}\n

:::

In addition, the SIMD instructions Neon for ARM can also be used in Go by the idea of this post. goat also supports ARM, and Gorse also implements vectorized computation libraries for Neon instructions (refer to github.com/gorse-io/gorse/base/floats). You are welcome to use goat to build your own vectorized functions in Go projects, more questions are welcome in Issues or Discord.

"},{"location":"backup/blog/frontend-in-go-modules/","title":"Integrate Frontend Artifacts via Go Modules","text":"

The ability to compile into a single binary is a great feature of the Go programming language, avoiding dependency management at deployment time. However, if the project contains front-end code, we need to find a way to embed the front-end artifact into the Go binary at compile time. The compilation process is as follows.

flowchart LR\n  frontend[Compile frontend code]-->embed[Embed to Go code]-->build[Compile Go code]
  1. Compile the front-end code.
  2. Convert front-end artifacts to Go embedding files. For example, the project rakyll/statik is able to embed any file into the Go source code. In addition, the Go language currently provides an official embedding solution embed that allows embedding to be done during compilation, eliminating step 2.
  3. Build Go code.

Usually front-end code and Go code are placed in the same repository or put into the Go repository through git submodule and compiled through Makefile or build scripts. But such a solution has a small drawback, we need to install the front-end toolchain when developing a Go project. For a project like Gorse, most of the development work is on the backend, and the front-end changes are very minimal. It is unnecessary to compile the front-end every time the repository is pulled, so a solution is needed to avoid the front-end compilation process during Go development. The Gorse project compiles the dashboard front-end code into a separate Go package that is integrated into the main repository via Go modules.

"},{"location":"backup/blog/frontend-in-go-modules/#compile-front-end-into-go-packages","title":"Compile Front-end into Go Packages","text":"

rakyll/statik can embed front-end artifacts into Go code, then it is perfectly possible to have the converted code referenced by Gorse as a separate Go package. Then after each modification of the front-end code:

  1. Compile the front-end code.
  2. Use rakyll/statik to convert front-end artifacts to Go embedding files and initialize Go packages with go mod init and go mod tidy.
  3. Push the Go package to the repository or branch used to store the artifact.

The above process can be written as a script, and since the Gorse front-end repository is hosted on GitHub, it can be automatically updated each time the front-end code is committed through GitHub Action.

name: build\non:\npush:\nbranches: [ master ] # Triggered when master commits code\njobs:\nbuild:\nruns-on: ubuntu-latest\nsteps:\n# Compile front-end code\n- uses: actions/checkout@v2\n- name: Install dependencies\nuses: borales/actions-yarn@v2.0.0\nwith:\ncmd: install\n- name: Build for production\nuses: borales/actions-yarn@v2.0.0\nwith:\ncmd: build\n# Embed into Go source code\n- name: Install Go\nuses: actions/setup-go@v2\n- name: Install statik & build embed files\nrun: |\nexport PATH=$PATH:$(go env GOPATH)/bin\ngo get github.com/rakyll/statik\nstatik -src=dist\n# Commit Go source code\n- name: Commit embed files\nrun: |\ncd ...\ngit clone https://github.com/gorse-io/dashboard.git dashboard-statik\ncd dashboard-statik\ngit config --local user.email \"41898282+github-actions[bot]@users.noreply.github.com\"\ngit config --local user.name \"github-actions[bot]\"\ngit checkout statik || git checkout --orphan statik\ngit rm -r --cached .\nrm statik.go go.mod go.sum\ncp -f ... /dashboard/statik/statik.go statik.go\ngo mod init github.com/gorse-io/dashboard\ngo mod tidy\ngit add statik.go go.mod go.sum\ngit commit -m \"Build embed files\"\ngit remote-set-url origin https://x-access-token:${{ secrets.GITHUB_TOKEN }}@github.com/${{ github.repository }}\ngit push origin statik\n

The above GitHub Action commits the Go code to the statik branch.

"},{"location":"backup/blog/frontend-in-go-modules/#using-front-end-artifacts-via-go-modules","title":"Using Front-end Artifacts via Go Modules","text":"

First, download the front-end artifact package. Since the product is stored on the statik branch, you need to add @statik after the package name.

go get -u github.com/gorse-io/dashboard@statik\n

Then, github.com/gorse-io/dashboard is imported and the front-end static files are saved in the filesystem created by fs.New().

import (\n\"github.com/rakyll/statik/fs\"\n_ \"github.com/gorse-io/dashboard\"\n)\n// ...\nstatikFS, err := fs.New()\nif err ! = nil {\nlog.Fatal(err)\n}\n// Example: Accessing the file system via HTTP\nhttp.Handle(\"/\", http.FileServer(statikFS))\nhttp.ListenAndServe(\":8080\", nil)\n
"},{"location":"backup/blog/frontend-in-go-modules/#summary","title":"Summary","text":"

Using Go modules to manage front-end artifacts is elegant, but only suitable for projects with few front-end changes. For projects where front-end and back-end development often needs joint debugging, such as mall projects, it is more appropriate to put the front-end and back-end code in the same repository.

"},{"location":"backup/reference/crossline/","title":"Crossline","text":"

I'm crossline

If you want to build a project, you can hire Scifabric for designing, building, and analysing your data. Check Scifabric solutions.

\ud83d\udca1 Note: If you want to build a project, you can hire Scifabric for designing, building, and analysing your data. Check Scifabric solutions.

I'm developing a cross-platfrom command line tool which can support autocomplete and history. Readline library is the first choice, but it can't use on Windows directly and you need to link the readline library explicitly. You also need to install the libreadline-dev package to build and if target machine version mismatches with build machine, the program can't run. The libreadline has over 30K LOC and has two so files: libreadline.so libhistory.so, and it depends on libncurses also.

introduction1

introduction2

introduction3

introduction4

introduction5

Crossline is a small, self-contained, zero-config, MIT licensed, cross-platform, readline and libedit replacement.

When should you use Crossline: * When you need a cross-platform readline: Windows, Linux, Unix, MacOS. * When you need a simple but versatile readline: more shortcuts, advanced search, simple autocomplete, paging, cusor APIs, color APIs, embedded help, etc. * When you need a customized readline: easy to extend. * When you need a small readline to build into program.

"},{"location":"backup/reference/crossline/#catalogue","title":"Catalogue","text":"
  • Features and Highlights
  • Background
  • Shortcuts
  • Multiple Line Mode
  • History Search
  • Crossline APIs
  • Color APIs
  • Paging APIs
  • Cursor APIs
  • Simple Example
  • SQL Parser Example
  • Keyboard Debug
  • Embedded Help
  • Extend Crossline
  • Customized Config
  • Build and Test
  • Related Projects
"},{"location":"backup/reference/crossline/#features-and-highlights","title":"Features and Highlights","text":"
  • Support many platforms: Windows, Linux, vt100 and xterm.
  • Support total 79 shortcuts and 40 functions.
  • Support most readline shortcuts (Emacs Standard bindings): move, edit, cut&paste, complete, history, control.
  • Support fast move between lines with Up Down and Ctrl/Alt+Up Ctrl/Alt+Down keys in multiple line edit mode.
  • Support some Windows command line shortcuts and extend some new convenient shortcuts.
  • Support history navigation, history show/clear and history save/load.
  • Support readline with initial input for editing.
  • Support autocomplete, key word help and syntax hints.
  • Support powerful interactive history search with multiple case insensitive including and excluding match patterns.
  • Support same edit shortcuts (except complete and history shortcuts) in history search mode.
  • Support color text for prompt, autocomplete, hints.
  • Support auto resizing when editing window/terminal size changed.
  • Support autocomplete, history show/search, help info paging.
  • Support cross platform paging control APIs.
  • Support cross platform cursor control APIs.
  • Support cross platform color control APIs.
  • Support convenient embedded F1 help in edit and history search mode, and you can call it anytime without losing current input.
  • Support convenient embedded Ctrl-^ keyboard debug mode to watch key code sequences.
  • Support Ctrl-C to exit edit and Ctrl-Z to suspend and resume job(Linux) in both edit and search mode.
  • Support pipe as input.
  • Pure C MIT license source code, no 3rd library dependency.
  • No any dynamic memory operations: malloc/free/realloc/new/delete/strdup/etc.
  • Very small only about 1200 LOC, and code logic is simple and easy to read.
  • Easy to customize your own shortcuts and new features.
  • Unicode is to be supported later.
"},{"location":"backup/reference/crossline/#background","title":"Background","text":"

I'm developing a cross-platfrom command line tool which can support autocomplete and history. Readline library is the first choice, but it can't use on Windows directly and you need to link the readline library explicitly. You also need to install the libreadline-dev package to build and if target machine version mismatches with build machine, the program can't run. The libreadline has over 30K LOC and has two so files: libreadline.so libhistory.so, and it depends on libncurses also.

Then I searched and found there's a small readline replacement linenoise written by Redis's author. Linenoise is used in Redis, MongoDB, and Android. It's very small (about 1,100 LOC), but it can only run on Linux and supports few shortcuts and features. Then I found there's a more powerful library linenoise-ng used in ArangoDB. It's based on MongoDB's linenoise which enhanced original linenoise to support Windows, more shortcuts, features and Unicode. But the code is embedded in MongoDB source code, then ArangoDB ported it out, did some improvements and made it an independent library.

At first I planned to use linenoise-ng, but I found it's still big (about 4,300 LOC) and complex. It uses C++ and C together, and C/C++ dynamic memory is used also which is there in original linenoise. I think it can be much simpler, then I did some prototype verification, and use a different method to implement this brand new, simple, pure C cross-platform enhanced readline replacement library.

"},{"location":"backup/reference/crossline/#shortcuts","title":"Shortcuts","text":"

Misc Commands

Shortcut Action F1 Show edit shortcuts help. Ctrl-^ Enter keyboard debugging mode.

Move Commands

Shortcut Action Ctrl-B, Left Move back a character. Ctrl-F, Right Move forward a character. Up Move cursor to up line. (For multiple lines) Ctrl-Up, Alt-Up, ESC+Up Move cursor to up line. (Ctrl-Up, Alt-Up only supports Windows/Xterm) Down Move cursor to down line. (Ctrl-Down, Alt-Down only support Windows/Xterm) Ctrl-Down, Alt-Down, ESC+Down Move cursor to down line. (Ctrl-Down, Alt-Down only support Windows/Xterm) Alt-B, Ctrl-Left, Alt-Left, ESC+Left Move back a word. (Ctrl-Left, Alt-Left only support Windows/Xterm) Alt-F, Ctrl-Right, Alt-Right, ESC+Right Move forward a word. (Ctrl-Right, Alt-Right only support Windows/Xterm) Ctrl-A, Home Move cursor to start of line. Ctrl-E, End Move cursor to end of line. Ctrl-L Clear screen and redisplay line.

Edit Commands

Shortcut Action Ctrl-H, Backspace Delete character before cursor. Ctrl-D, DEL Delete character under cursor. Alt-U Uppercase current or following word. Alt-L Lowercase current or following word. Alt-C Capitalize current or following word. Alt-\\ Delete whitespace around cursor. Ctrl-T Transpose previous character with current character.

Cut&Paste Commands

Shortcut Action Ctrl-K, Ctrl-End, Alt-End, ESC+End Cut from cursor to end of line. (Ctrl-End, Alt-End only support Windows/Xterm) Ctrl-U, Ctrl-Home, Alt-Home, ESC+Home Cut from start of line to cursor. (Ctrl-Home, Alt-Home only support Windows/Xterm) Ctrl-X Cut whole line. Alt-Backspace, Clt-Backspace, Esc+Backspace Cut word to left of cursor. (Clt-Backspace only supports Windows/Xterm) Alt-D, Alt-Del, Ctrl-Del, ESC+Del Cut word following cursor. (Alt-Del,Ctrl-Del only support Windows/Xterm) Ctrl-W Cut to left till whitespace (not word). Ctrl-Y, Ctrl-V, Insert Paste last cut text.

Complete Commands

Shortcut Action TAB, Ctrl-I Autocomplete. Alt-=, Alt-? List possible completions.

History Commands

Shortcut Action Ctrl-P, Up Fetch previous line in history. (Up works when cursor in first line or end of last line) Ctrl-N, Down Fetch next line in history. (Down works when cursor in last line) Alt-<, PgUp Move to first line in history. Alt->, PgDn Move to end of input history. Ctrl-R, Ctrl-S Search history. F4 Search history with current input. F1 Show search help when in search mode. F2 Show history. F3 Clear history (need confirm).

Control Commands

Shortcut Action Enter, Ctrl-J, Ctrl-M EOL and accept line. Ctrl-C, Ctrl-G EOF and abort line. Ctrl-D EOF if line is empty. Alt-R Revert line. Undo all changes made to this line. Ctrl-Z Suspend Job. (Linux only, fg will resume edit)

Notes

  • In multiple lines, Up/Down can move between lines. Up will fetch history when in first line or end of last line(for quick history move), and Down will fetch history when in last line.
  • For Windows and xterm, almost all shortcuts are supported.
  • For some terminal tools you need to enable Alt as meta key. SecureCRT: check Terminal->Emulation->Emacs->use ALT as meta key
  • Backspace key is 8 or 127, and Del key should use Escape code. SecurCRT: check Terminal->Emulation->Mapped Keys->Backspace sends delete
  • For vt100 and Linux terminals, many Alt-key doesn't work, and an alternate way is to press ESC first then press key, see above ESC+Key. Putty can send Alt as ESC, so no need to do this way.
  • Ctrl+S is readline shortcut to search history also, but it'll halt terminal, so don't use it with Linux system, use Ctrl+Q to exit freezing state if pressing by mistake.

Terminal Limitations

  • Some terminals only support left Alt.
  • Linux console doesn't support: Alt-?, Alt-<, Alt->.
  • SecureCRT vt100 doesn't support: Home, End, Del, Insert, PgUp, PgDn.
  • SecureCRT xterm doesn't support: Ctrl-Home, Alt-Home, Ctrl-End, Alt-End, Alt-Del, Ctrl-Del, Clt-Backspace.

Goto Top

"},{"location":"backup/reference/crossline/#multiple-line-mode","title":"Multiple Line Mode","text":"

Crossline supports multiple line edit mode like readline. If you input a very long line and then you want to edit some part many lines above, it's not convenient to move to the position quickly even with Ctrl+Left. I used to copy input text to a GUI editor like notepad++, then edit and copy back.

To move quickly in multiple lines, I implemented a multiple line edit mode like other editors and can use Ctrl/Alt+Up Ctrl/Alt+Down to move between lines quickly. You can also use Up Down key and to keep the original history shortcuts, the key behavior will behave as following:

  • For single line, Up Down key are history shortcuts.
  • In first line of multiple lines, Down key will move to below line, Up key will fetch previous history.
  • In middle line of multiple lines, Down key will move to below line, Up key will move to above line.
  • In last line(cursor not at end) of multiple lines, Down key will fetch next history, Up key will move to above line.
  • In last line(cursor at end) of multiple lines, Up Down key are history shortcuts. This is to keep navigating history quickly.
  • Ctrl-P Ctrl-N history shortcuts are not changed.

So if you want to use Up Down to move quickly between lines, make sure the cursor is not at end of last line, and if you want to navigate history quickly with Up Down, make sure the cursor is at end of last line in multiple lines.

"},{"location":"backup/reference/crossline/#history-search","title":"History Search","text":"

Original readline supports incremental search(Ctrl-R,Ctrl-S) and none-incremental search(Alt-N,Alt-P). I tried both and think they're not convenient or efficient to use, so I implemented a brand new interactive search method.

Enter interactive history search mode

  • Ctrl-R: Search history.
  • F4: Search history with current input as search patterns.

Exit interactive history search mode

  • Ctrl-C: You can exit search mode anytime and keep your original input.
  • Ctrl-D: On empty line only.

Input patterns

  • You can use all edit shortcuts except complete and history shortcuts.
  • You can use Insert,Ctrl-Y,Ctrl-S to paste last search patterns.
  • If nothing input will show all history (F2 can show history too).

Patterns syntax

You can press F1 to get help. Patterns are separated by ' ', patter match is case insensitive. * select: choose line including select * -select: choose line excluding select * \"select from\": choose line including select from * -\"select from\": choose line excluding select from * \"select from\" where -\"order by\" -limit : choose line including select from and where and excluding order by or limit

Select history

  • If only one history is found, id 1 is provided automatically, and you just press Enter to select.
  • You can press Ctrl-C to skip choosing.
  • You can press Enter or Ctrl-D on empty line to skip choosing.

Example

SQL> <F2> // show history\n   1  hello world\n   2  select from user\n   3  from select table\n   4  SELECT from student\n   5  Select from teacher\n\nSQL> <Ctrl+R>\nInput Patterns <F1> help: select from\n   1  select from user\n   2  from select table\n   3  SELECT from student\n   4  Select from teacher\nInput history id: 3\nSQL> SELECT from student<Alt+R> // Revert line\n\nSQL> <F4>\nInput Patterns <F1> help: <Insert> // paste last history pattern: select from\nInput Patterns <F1> help: \"select from\"\n   1  select from user\n   2  SELECT from student\n   3  Select from teacher\nInput history id: 3\nSQL> Select from teacher<Alt+R>\n\nSQL> SELECT from<F4> // search with pattern: Select from\nInput Patterns <F1> help: SELECT from\nInput Patterns <F1> help: \"SELECT from\" -user -teacher\n   1  SELECT from student\nInput history id: 1\nSQL> SELECT from student\n

Goto Top

"},{"location":"backup/reference/crossline/#crossline-apis","title":"Crossline APIs","text":"
// Main API to read a line, return buf if get line, return NULL if EOF.\nchar* crossline_readline (const char *prompt, char *buf, int size);\n// Same with crossline_readline except buf holding initial input for editing.\nchar* crossline_readline2 (const char *prompt, char *buf, int size);\n// Set move/cut word delimiter, default is all not digital and alphabetic characters.\nvoid  crossline_delimiter_set (const char *delim);\n// Read a character from terminal without echo\nint     crossline_getch (void);\n
  • History APIs
// Save history to file\nint   crossline_history_save (const char *filename);\n// Load history from file\nint   crossline_history_load (const char *filename);\n// Show history in buffer\nvoid  crossline_history_show (void);\n// Clear history\nvoid  crossline_history_clear (void);\n
  • Completion APIs
typedef struct crossline_completions_t crossline_completions_t;\ntypedef void (*crossline_completion_callback) (const char *buf, crossline_completions_t *pCompletions);\n// Register completion callback\nvoid  crossline_completion_register (crossline_completion_callback pCbFunc);\n// Add completion in callback. Word is must, help for word is optional.\nvoid  crossline_completion_add (crossline_completions_t *pCompletions, const char *word, const char *help);\n// Set syntax hints in callback\nvoid  crossline_hints_set (crossline_completions_t *pCompletions, const char *hints);\n
"},{"location":"backup/reference/crossline/#color-apis","title":"Color APIs","text":"

Color definitions

Color Note CROSSLINE_FGCOLOR_DEFAULT If not set, it's the default value. CROSSLINE_FGCOLOR_BLACK Set text color to black. \\t is not supported in Linux terminal, same below CROSSLINE_FGCOLOR_RED Set text color to red. CROSSLINE_FGCOLOR_GREEN Set text color to green. CROSSLINE_FGCOLOR_YELLOW Set text color to yellow. CROSSLINE_FGCOLOR_BLUE Set text color to blue. CROSSLINE_FGCOLOR_MAGENTA Set text color to magenta. CROSSLINE_FGCOLOR_CYAN Set text color to cyan. CROSSLINE_FGCOLOR_WHITE Set text color to white. CROSSLINE_FGCOLOR_BRIGHT Set text color bright. CROSSLINE_BGCOLOR_DEFAULT If not set, it's the default value. CROSSLINE_BGCOLOR_BLACK Set background color to black. \\t is not supported in Linux terminal, same below. Don't use \\n in Linux terminal, same below. CROSSLINE_BGCOLOR_RED Set background color to red. CROSSLINE_BGCOLOR_GREEN Set background color to green. CROSSLINE_BGCOLOR_YELLOW Set background color to yellow. CROSSLINE_BGCOLOR_BLUE Set background color to blue. CROSSLINE_BGCOLOR_MAGENTA Set background color to magenta. CROSSLINE_BGCOLOR_CYAN Set background color to cyan. CROSSLINE_BGCOLOR_WHITE Set background color to white. CROSSLINE_BGCOLOR_BRIGHT Set background color bright. CROSSLINE_UNDERLINE Set text to underline style. CROSSLINE_COLOR_DEFAULT If not set, it's the default value.

CROSSLINE_FGCOLOR_DEFAULT, CROSSLINE_BGCOLOR_DEFAULT, CROSSLINE_COLOR_DEFAULT are all zero, so it's fine to leave without it or just give 0.

Color APIs

// Set text color, CROSSLINE_COLOR_DEFAULT will revert to default setting\n// `\\t` is not supported in Linux terminal, same below. Don't use `\\n` in Linux terminal, same below.\nvoid crossline_color_set (crossline_color_e color);\n// Set default prompt color\nvoid crossline_prompt_color_set (crossline_color_e color);\n// Add completion with color.\nvoid  crossline_completion_add_color (crossline_completions_t *pCompletions, const char *word, crossline_color_e wcolor, const char *help, crossline_color_e hcolor);\n// Set syntax hints with color\nvoid  crossline_hints_set_color (crossline_completions_t *pCompletions, const char *hints, crossline_color_e color);\n

Color Example

You can get the color example code in example2.c color_test()

Goto Top

"},{"location":"backup/reference/crossline/#paging-apis","title":"Paging APIs","text":"

These APIs are used internally first, then I think they're common and can be used by CLI tools also, so make them open.

// Reset paging before starting paging control\nvoid crossline_paging_reset (void);\n// Check paging after print a line, return 1 means quit, 0 means continue\n// if you know only one line is printed, just give line_len = 1\nint  crossline_paging_check (int line_len);\n

Code in example2.c 

static void pagint_test ()\n{\nint i;\ncrossline_paging_reset ();\nfor (i = 0; i < 256; ++i) {\nprintf (\"Paging test: %3d\\n\", i);\nif (crossline_paging_check (sizeof(\"paging test: \") + 3)) {\nbreak;\n}\n}\n}\n

It'll work as following:

Crossline> paging\nRead line: \"paging\"\nPaging test:   0\nPaging test:   1\nPaging test:   2\nPaging test:   3\nPaging test:   4\nPaging test:   5\nPaging test:   6\n*** Press <Space> or <Enter> to continue . . .\n
"},{"location":"backup/reference/crossline/#cursor-apis","title":"Cursor APIs","text":"

These APIs are used internally first, then I think they're common and can be used by CLI tools also, so make them open and write some APIs which are not used by crossline at all.

// Get screen rows and columns\nvoid crossline_screen_get (int *pRows, int *pCols);\n// Clear current screen\nvoid crossline_screen_clear (void);\n// Get cursor postion (0 based)\nint  crossline_cursor_get (int *pRow, int *pCol);\n// Set cursor postion (0 based)\nvoid crossline_cursor_set (int row, int col);\n/* Move cursor with row and column offset, \n *   row_off > 0 move up row_off lines, \n *           < 0 move down abs(row_off) lines\n *           = 0 no move for row, \n *   similar with col_off\n */\nvoid crossline_cursor_move (int row_off, int col_off);\n// Hide or show cursor\nvoid crossline_cursor_hide (int bHide);\n

Goto Top

"},{"location":"backup/reference/crossline/#simple-example","title":"Simple Example","text":"

example.c is a very simple example to use crossline.

static void completion_hook (char const *buf, crossline_completions_t *pCompletion)\n{\nint i;\nstatic const char *cmd[] = {\"insert\", \"select\", \"update\", \"delete\", \"create\", \"drop\", \"show\", \"describe\", \"help\", \"exit\", \"history\", \"insert\", \"select\", \"update\", \"delete\", \"create\", \"drop\", \"show\", \"describe\", \"help\", \"exit\", \"history\", NULL};\nfor (i = 0; NULL != cmd[i]; ++i) {\nif (0 == strncmp(buf, cmd[i], strlen(buf))) {\ncrossline_completion_add (pCompletion, cmd[i], NULL);\n}\n}\n}\nint main ()\n{\nchar buf[256];\ncrossline_completion_register (completion_hook);\ncrossline_history_load (\"history.txt\");\nwhile (NULL != crossline_readline (\"Crossline> \", buf, sizeof(buf))) {\nprintf (\"Read line: \\\"%s\\\"\\n\", buf);\n}    crossline_history_save (\"history.txt\");\nreturn 0;\n}\n

example2.c is an enhanced example, which supports color, color test, paging test and have the edit with initial input example.

"},{"location":"backup/reference/crossline/#sql-parser-example","title":"SQL Parser Example","text":"

example_sql.c, this example implements a simple SQL syntax parser with following syntax format, please read code for details.

INSERT INTO <table> SET column1=value1,column2=value2,...\nSELECT <* | column1,columnm2,...> FROM <table> [WHERE] [ORDER BY] [LIMIT] [OFFSET]\nUPDATE <table> SET column1=value1,column2=value2 [WHERE] [ORDER BY] [LIMIT] [OFFSET]\nDELETE FROM <table> [WHERE] [ORDER BY] [LIMIT] [OFFSET]\nCREATE [UNIQUE] INDEX <name> ON <table> (column1,column2,...)\nDROP {TABLE | INDEX} <name>\nSHOW {TABLES | DATABASES}\nDESCRIBE <TABLE>\nhelp {INSERT | SELECT | UPDATE | DELETE | CREATE | DROP | SHOW | DESCRIBE | help | exit | history}\n
You can use this example to practice the shortcuts above.

SQL> <TAB>  // show autocomplete words and help\nINSERT      Insert a record to table\nSELECT      Select records from table\nUPDATE      Update records in table\nDELETE      Delete records from table\nCREATE      Create index on table\nDROP        Drop index or table\nSHOW        Show tables or databases\nDESCRIBE    Show table schema\nhelp        Show help for Topic\nexit        Exit shell\nhistory     Show history\n*** Press <Space> or <Enter> to continue . . .\nSQL> help <TAB> // show autocomplete words list\nINSERT    SELECT    UPDATE    DELETE    CREATE    DROP      SHOW\nDESCRIBE  help      exit      history\nSQL> CREATE INDEX <TAB> // show autocomplete hints\nPlease input: index name\n

Goto Top

"},{"location":"backup/reference/crossline/#keyboard-debug","title":"Keyboard Debug","text":"

Enter keyboard debug mode

Press Ctrl-^ to enter keyboard debug mode, then you can type any key or composite key, and the code sequence will be displaced. This can be used to discover new key code sequences or debug especially for Linux system with different terminals. Note: For Windows, Alt key is a state, so it's not displayed.

Exit keyboard debug mode

Press Ctrl-C to exit.

Example

SQL> <Ctrl-^>\nEnter keyboard debug mode, <Ctrl-C> to exit debug\n 27 0x1b ( )\n 91 0x5b ([)\n 65 0x41 (A)\n 31 0x1f ( )\n 27 0x1b ( )\n 91 0x5b ([)\n 72 0x48 (H)\n
"},{"location":"backup/reference/crossline/#embedded-help","title":"Embedded Help","text":"

Edit mode

SQL> <F1>\n Misc Commands\n +-------------------------+--------------------------------------------------+\n | F1                      |  Show edit shortcuts help.                       |\n | Ctrl-^                  |  Enter keyboard debugging mode.                  |\n +-------------------------+--------------------------------------------------+\n Move Commands\n +-------------------------+--------------------------------------------------+\n | Ctrl-B, Left            |  Move back a character.                          |\n | Ctrl-F, Right           |  Move forward a character.                       |\n | Up, ESC+Up              |  Move cursor to up line. (For multiple lines)    |\n |   Ctrl-Up, Alt-Up       |  (Ctrl-Up, Alt-Up only supports Windows/Xterm)   |\n | Down, ESC+Down          |  Move cursor to down line. (For multiple lines)  |\n |   Ctrl-Down,Alt-Down    |  (Ctrl-Down, Alt-Down only support Windows/Xterm)|\n | Alt-B, ESC+Left,        |  Move back a word.                               |\n |   Ctrl-Left, Alt-Left   |  (Ctrl-Left, Alt-Left only support Windows/Xterm)|\n*** Press <Space> or <Enter> to continue . . .\n

Search mode

SQL> <F4> or <Ctrl+R>\nInput Patterns <F1> help: <F1>\nPatterns are separated by ' ', patter match is case insensitive:\n    select:   choose line including 'select'\n    -select:  choose line excluding 'select'\n    \"select from\":  choose line including \"select from\"\n    -\"select from\": choose line excluding \"select from\"\nExample:\n    \"select from\" where -\"order by\" -limit:\n         choose line including \"select from\" and 'where'\n         and excluding \"order by\" or 'limit'\n
"},{"location":"backup/reference/crossline/#extend-crossline","title":"Extend Crossline","text":"

You can extend crossline to add new shortcuts and action easily.

Use keyboard debug mode to find the key code sequences

  • For Windows, most key has only one code, some function key has 2, first is 224 or 0, then follows another code. Alt key in Windows is a state, please refer crossline_getkey.
  • For Linux system, most function key has an escape sequences(up to 6), and one function key may have many different escape sequences for different terminal modes, so you need to test case by case.

Define new key

  • For Ctrl-key, most are single code, and you can use CTRL_KEY(key) directly.
  • For windows, most Alt-key can use ALT_KEY(num).
  • For Linux, most Alt key is escape sequences, and you can use existing macro like ESC_KEY6, example: KEY_ALT_LEFT = ESC_KEY6('1','3','D'), // xterm Esc[1;3D: Move back a word.

Support new Esc+key

Please add the conversion in crossline_key_esc2alt

Map new key to existing action

  • If it's a new different escape sequences for same key, please map the new key to main action key in crossline_key_mapping.
  • If it's a new key, please add case to crossline_readline_input.

Add new action

Please add case and action code in crossline_readline_input. You can refer existing case of similar action to write your new action. Use crossline_refreash to print line after updating buf.

Debug your code

It's not easy to debug with print or debugger tools like GDB for terminal tools, and you can use crossline_debug(\"xxx\",...) to print debug information to file crossline_debug.txt.

Goto Top

"},{"location":"backup/reference/crossline/#customized-config","title":"Customized Config","text":"

Word delimiters for move and cut

Default is defined by CROSS_DFT_DELIMITER. 

#define CROSS_DFT_DELIMITER            \" !\\\"#$%&'()*+,-./:;<=>?@[\\\\]^_`{|}~\"\n
You can modify it or use crossline_delimiter_set to change it.

History

The history line len can be less than user buf len, and it'll cut to history line len when storing to history buf. 

#define CROSS_HISTORY_MAX_LINE       256         // Maximum history line number\n#define CROSS_HISTORY_BUF_LEN        4096        // History line length\n#define CROSS_HIS_MATCH_PAT_NUM      16          // History search pattern number\n

Completion 

#define CROSS_COMPLET_MAX_LINE        512        // Maximum completion word number\n#define CROSS_COMPLET_WORD_LEN        64         // Completion word length\n#define CROSS_COMPLET_HELP_LEN        256        // Completion word's help length\n#define CROSS_COMPLET_HINT_LEN        128        // Completion syntax hints length\n

"},{"location":"backup/reference/crossline/#build-and-test","title":"Build and Test","text":"

On Windows, you can add the source code to a Visual Studio project to build or enter Tools Command Prompt for VS from menu to build in command line which is more efficient.

Windows MSVC

cl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example.c /Feexample.exe\ncl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example2.c /Feexample2.exe\ncl -D_CRT_SECURE_NO_WARNINGS -W4 User32.Lib crossline.c example_sql.c /Feexample_sql.exe\n

Windows Clang

clang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example.c -o example.exe\nclang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example2.c -o example2.exe\nclang -D_CRT_SECURE_NO_WARNINGS -Wall -lUser32 crossline.c example_sql.c -o example_sql.exe\n

GCC(Linux, MinGW, Cygwin, MSYS2)

gcc -Wall crossline.c example.c -o example\ngcc -Wall crossline.c example2.c -o example2\ngcc -Wall crossline.c example_sql.c -o example_sql\n

Linux Clang

clang -Wall crossline.c example.c -o example\nclang -Wall crossline.c example2.c -o example2\nclang -Wall crossline.c example_sql.c -o example_sql\n
"},{"location":"backup/reference/crossline/#related-projects","title":"Related Projects","text":"
  • Linenoise a small self-contained alternative to readline and libedit
  • Linenoise-ng is a fork of Linenoise that aims to add more advanced features like UTF-8 support, Windows support and other features. Uses C++ instead of C as development language.

Goto Top

"},{"location":"backup/reference/markdown/","title":"Markdown Cheat Sheet","text":"

Thanks for visiting The Markdown Guide!

This Markdown cheat sheet provides a quick overview of all the Markdown syntax elements. It can\u2019t cover every edge case, so if you need more information about any of these elements, refer to the reference guides for basic syntax and extended syntax.

"},{"location":"backup/reference/markdown/#basic-syntax","title":"Basic Syntax","text":"

These are the elements outlined in John Gruber\u2019s original design document. All Markdown applications support these elements.

"},{"location":"backup/reference/markdown/#heading","title":"Heading","text":""},{"location":"backup/reference/markdown/#h1","title":"H1","text":""},{"location":"backup/reference/markdown/#h2","title":"H2","text":""},{"location":"backup/reference/markdown/#h3","title":"H3","text":""},{"location":"backup/reference/markdown/#bold","title":"Bold","text":"

bold text

"},{"location":"backup/reference/markdown/#italic","title":"Italic","text":"

italicized text

bold italicized text

italicized text

italicized text

"},{"location":"backup/reference/markdown/#line-breaks","title":"Line Breaks","text":"

First line with two spaces after. And the next line.

First line with the HTML tag after. And the next line.

"},{"location":"backup/reference/markdown/#blockquote","title":"Blockquote","text":"

blockquote

"},{"location":"backup/reference/markdown/#blockquotes-with-other-elements","title":"Blockquotes with Other Elements","text":""},{"location":"backup/reference/markdown/#the-quarterly-results-look-great","title":"The quarterly results look great!","text":"
  • Revenue was off the chart.
  • Profits were higher than ever.

Everything is going according to plan.

"},{"location":"backup/reference/markdown/#ordered-list","title":"Ordered List","text":"
  1. First item
  2. Second item
  3. Third item
    1. Indented item
    2. Indented item
  4. Fourth item
"},{"location":"backup/reference/markdown/#unordered-list","title":"Unordered List","text":"
  • First item
  • Second item
  • Third item
    • Indented item
    • Indented item
  • Fourth item

Mix

  1. First item
  2. Second item
  3. Third item
    • Indented item
    • Indented item

  4. Fourth item

  5. This is the first list item.

  6. Here's the second list item.

    I need to add another paragraph below the second list item.

    I need to add another paragraph below the second list item.

  7. And here's the third list item.

"},{"location":"backup/reference/markdown/#code","title":"Code","text":"

code

"},{"location":"backup/reference/markdown/#horizontal-rule","title":"Horizontal Rule","text":""},{"location":"backup/reference/markdown/#link","title":"Link","text":"

Markdown Guide

Link

Link

Markdown Guide2

can't have tab

"},{"location":"backup/reference/markdown/#formatting-links","title":"Formatting Links","text":"

I love supporting the EFF. This is the Markdown Guide. See the section on code.

"},{"location":"backup/reference/markdown/#urls-and-email-addresses","title":"URLs and Email Addresses","text":"

https://www.markdownguide.org fake@example.com

"},{"location":"backup/reference/markdown/#image","title":"Image","text":"

abs img

rel img

Std img

Image caption

new fig

gif

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla et euismod nulla. Curabitur feugiat, tortor non consequat finibus, justo purus auctor massa, nec semper lorem quam in massa.

image in left image in left image in left image in left image in left image in left image in left image in left

"},{"location":"backup/reference/markdown/#escaping-characters","title":"Escaping Characters","text":"

* `

"},{"location":"backup/reference/markdown/#extended-syntax","title":"Extended Syntax","text":"

These elements extend the basic syntax by adding additional features. Not all Markdown applications support these elements.

"},{"location":"backup/reference/markdown/#table","title":"Table","text":"Syntax Description Header Title Paragraph Text Syntax Description Test Text Header Title Here's this Paragraph Text And more Method Description GET Fetch resource PUT Update resource DELETE Delete resource"},{"location":"backup/reference/markdown/#fenced-code-block","title":"Fenced Code Block","text":"
{\n\"firstName\": \"John\",\n\"lastName\": \"Smith\",\n\"age\": 25\n}\n
"},{"location":"backup/reference/markdown/#footnote","title":"Footnote","text":"

Here's a sentence with a footnote. 1

Here's a sentence with a footnote. 1

"},{"location":"backup/reference/markdown/#heading-id","title":"Heading ID","text":""},{"location":"backup/reference/markdown/#custom-id","title":"My Great Heading","text":""},{"location":"backup/reference/markdown/#definition-list","title":"Definition List","text":"First Term This is the definition of the first term. Second Term This is one definition of the second term. This is another definition of the second term."},{"location":"backup/reference/markdown/#strikethrough","title":"Strikethrough","text":"

The world is flat.

"},{"location":"backup/reference/markdown/#task-list","title":"Task List","text":"
  • Write the press release
  • Update the website
  • Contact the media
"},{"location":"backup/reference/markdown/#emoji","title":"Emoji","text":"

\u2764\ufe0f That is so funny!

(See also Copying and Pasting Emoji)

"},{"location":"backup/reference/markdown/#highlight","title":"Highlight","text":"

I need to highlight these very important words.

"},{"location":"backup/reference/markdown/#subscript","title":"Subscript","text":"

H2O

"},{"location":"backup/reference/markdown/#superscript","title":"Superscript","text":"

X2

"},{"location":"backup/reference/markdown/#automatic-url-linking","title":"Automatic URL Linking","text":"

http://www.example.com

http://www.example.com

"},{"location":"backup/reference/markdown/#admonition","title":"Admonition","text":"

I'm Note

Some text here

Addtional Content

I'm Warning

Some text here

Addtional Content

The End

  1. This is the footnote.\u00a0\u21a9\u21a9

"},{"location":"blog/why-new-embdb/","title":"Why Need a New Embedded Database","text":"

TBD

"},{"location":"blog/benchmark/crossdb-vs-lmdb/","title":"CrossDB vs LMDB Benchmark","text":"

Test tool: CrossBench

DB Driver: LMDB CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core

"},{"location":"blog/benchmark/crossdb-vs-lmdb/#test-server","title":"Test Server","text":"
CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB\nHDD         : DELL PERC H755 Front SCSI Disk\nOS          : Ubuntu 20.04\nLMDB        : v0.9.24\nCrossDB     : v0.5.0\n
"},{"location":"blog/benchmark/crossdb-vs-lmdb/#lmdb-config","title":"LMDB Config","text":"
MDB_NOSYNC | MDB_NOMETASYNC | MDB_WRITEMAP | MDB_MAPASYNC | MDB_NORDAHEAD\n
"},{"location":"blog/benchmark/crossdb-vs-lmdb/#on-disk-database-benchmark","title":"On-Disk Database Benchmark","text":"

Test Script

CrossDBLMDB 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s d -i 1k   -q 30m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 10k  -q 30m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 100k -q 20m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 1m   -q 10m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 10m  -q 5m  -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 100m -q 5m  -u 1m  -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./lmdb-bench.bin -H -r 0\nfor i in $loop; do ./lmdb-bench.bin -s d -i 1k   -q 10m -u 5m -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s d -i 10k  -q 10m -u 5m -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s d -i 100k -q 10m -u 5m -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s d -i 1m   -q 5m  -u 5m -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s d -i 10m  -q 2m  -u 2m -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s d -i 100m -q 1m  -u 1m -Q -H -c $cpu; done\n

Small Data Set Test

Note

LMDB DB size is fixed and preallocated, so it's faster is 1K insert test.

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-lmdb/#on-ramdisk-database-benchmark","title":"On-RamDisk Database Benchmark","text":"

Test Script

CrossDBLMDB 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s r -i 1k   -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 10k  -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 100k -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./lmdb-bench.bin -H -r 0\nfor i in $loop; do ./lmdb-bench.bin -s r -i 1k   -q 10m -u 5m  -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s r -i 10k  -q 10m -u 5m  -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s r -i 100k -q 10m -u 5m  -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s r -i 1m   -q 5m  -u 5m  -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s r -i 10m  -q 2m  -u 2m  -Q -H -c $cpu; done\nfor i in $loop; do ./lmdb-bench.bin -s r -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-sqlite3/","title":"CrossDB vs SQLite3 Benchmark","text":"

Test tool: CrossBench

DB Driver: SQLite3 CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core

Note

The SQLite DB driver tries to tune SQLite performance as fast as possible and uses sqlite3_prepare_v2 to precompile SQL statements. If you have better optimization methods and it works, please let us know.

"},{"location":"blog/benchmark/crossdb-vs-sqlite3/#sqlite3-config","title":"SQLite3 Config","text":"On-DiskRamDiskIn-Memory 
PRAGMA synchronous = NORMAL\nPRAGMA journal_mode = WAL\nPRAGMA temp_store = memory\nPRAGMA optimize\n
PRAGMA synchronous = OFF\nPRAGMA journal_mode = WAL\nPRAGMA temp_store = memory\nPRAGMA optimize\n
PRAGMA synchronous = OFF\nPRAGMA journal_mode = OFF\nPRAGMA temp_store = memory\nPRAGMA optimize\n
"},{"location":"blog/benchmark/crossdb-vs-sqlite3/#test-server","title":"Test Server","text":"
CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB\nHDD         : DELL PERC H755 Front SCSI Disk\nOS          : Ubuntu 20.04\nSQLite3     : v3.31.1\nCrossDB     : v0.5.0\n
"},{"location":"blog/benchmark/crossdb-vs-sqlite3/#on-disk-database-benchmark","title":"On-Disk Database Benchmark","text":"

Test Script

CrossDBSQLite3 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s d -i 1k   -q 30m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 10k  -q 30m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 100k -q 20m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 1m   -q 10m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 10m  -q 5m  -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s d -i 100m -q 5m  -u 1m  -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./sqlite-bench.bin -H -r 0\nfor i in $loop; do ./sqlite-bench.bin -s d -i 1k   -q 1k  -u 1k  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s d -i 10k  -q 1m  -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s d -i 100k -q 1m  -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s d -i 1m   -q 1m  -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s d -i 10m  -q 1m  -u 1m  -Q -H -c $cpu; done\n# This test is very very very slow\nfor i in $loop; do ./sqlite-bench.bin -s d -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-sqlite3/#on-ramdisk-database-benchmark","title":"On-RamDisk Database Benchmark","text":"

Test Script

CrossDBSQLite3 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s r -i 1k   -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 10k  -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 100k -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s r -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./sqlite-bench.bin -H -r 0\nfor i in $loop; do ./sqlite-bench.bin -s r -i 1k   -q 1m -u 500k  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s r -i 10k  -q 1m -u 500k  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s r -i 100k -q 1m -u 300k  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s r -i 1m   -q 500k -u 250k -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s r -i 10m  -q 500k -u 250k -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s r -i 100m -q 500k -u 250k -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-sqlite3/#in-memory-database-benchmark","title":"In-Memory Database Benchmark","text":"

Test Script

CrossDBSQLite3 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./sqlite-bench.bin -H -r 0\nfor i in $loop; do ./sqlite-bench.bin -s m -i 1k   -q 2m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s m -i 10k  -q 2m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s m -i 100k -q 2m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s m -i 1m   -q 1m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s m -i 10m  -q 1m -u 1m  -Q -H -c $cpu; done\nfor i in $loop; do ./sqlite-bench.bin -s m -i 100m -q 1m -u 1m  -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-stlhmap/","title":"CrossDB vs STL Hashmap Benchmark","text":"

Test tool: CrossBench

DB Driver: STLHashmap CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core, To make test fair, STL map uses read-write lock as CrossDB is thread-safe with read-write lock also.

Note

Official STL Hashmap is unordered_map, which is the fastest data container in CPP. This test is to show how close CrossDB performance to STL Hashmap and CrossDB query can be faster than Hashmap in large data set test.

"},{"location":"blog/benchmark/crossdb-vs-stlhmap/#test-server","title":"Test Server","text":"
CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB\nHDD         : DELL PERC H755 Front SCSI Disk\nOS          : Ubuntu 20.04\nSTL         : GCC v10.2.1\nCrossDB     : v0.5.0\n
"},{"location":"blog/benchmark/crossdb-vs-stlhmap/#in-memory-database-benchmark","title":"In-Memory Database Benchmark","text":"

Test Script

CrossDBSTL Hashmap 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./stlhmap-bench.bin -H -r 0\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 1k   -q 60m -u 60m -Q -H -c $cpu; done\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 10k  -q 60m -u 60m -Q -H -c $cpu; done\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 100k -q 30m -u 40m -Q -H -c $cpu; done\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./stlhmap-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"blog/benchmark/crossdb-vs-stlmap/","title":"CrossDB vs STL Map Benchmark","text":"

Test tool: CrossBench

DB Driver: STLMap CrossDB

Test Config: Random Access, Single Thread, Bind CPU Core, To make test fair, STL map uses read-write lock as CrossDB is thread-safe with read-write lock also.

"},{"location":"blog/benchmark/crossdb-vs-stlmap/#test-server","title":"Test Server","text":"
CPU         : Intel(R) Xeon(R) Gold 5318Y CPU @ 2.10GHz cache size 36864 KB\nHDD         : DELL PERC H755 Front SCSI Disk\nOS          : Ubuntu 20.04\nSTL         : GCC v10.2.1\nCrossDB     : v0.5.0\n
"},{"location":"blog/benchmark/crossdb-vs-stlmap/#in-memory-database-benchmark","title":"In-Memory Database Benchmark","text":"

Test Script

CrossDBSTL Map 
loop=\"1 2 3\"\n./crossdb-bench.bin -H -r 0\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1k   -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10k  -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100k -q 40m -u 30m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 1m   -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 10m  -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./crossdb-bench.bin -s m -i 100m -q 10m -u 10m -Q -H -c $cpu; done\n
loop=\"1 2 3\"\n./stlmap-bench.bin -H -r 0\nfor i in $loop; do ./stlmap-bench.bin -s m -i 1k   -q 40m -u 40m -Q -H -c $cpu; done\nfor i in $loop; do ./stlmap-bench.bin -s m -i 10k  -q 20m -u 20m -Q -H -c $cpu; done\nfor i in $loop; do ./stlmap-bench.bin -s m -i 100k -q 10m -u 10m -Q -H -c $cpu; done\nfor i in $loop; do ./stlmap-bench.bin -s m -i 1m   -q 5m  -u 5m  -Q -H -c $cpu; done\nfor i in $loop; do ./stlmap-bench.bin -s m -i 10m  -q 2m  -u 2m  -Q -H -c $cpu; done\nfor i in $loop; do ./stlmap-bench.bin -s m -i 100m -q 1m  -u 1m  -Q -H -c $cpu; done\n

Small Data Set Test

Large Data Set Test

"},{"location":"docs/get-started/","title":"Get Started","text":""},{"location":"docs/get-started/#download","title":"Download","text":"

Choose the OS and CPU, then Download here.

Linux/FreeBSD package file list

File Descritpion crossdb.h The only header file libcrossdb.so The only shared library crossdb-cli CrossDB command line tool examples/ CrossDB example code

Windows package file list

File Descritpion crossdb.h The only header file crossdb.dll The only shared library crossdb.lib For build with MSCV compiler crossdb-cli.exe CrossDB command line tool examples/ CrossDB example code

MacOS package file list

File Descritpion crossdb.h The only header file libcrossdb.dylib The only shared library for X64 and AMD64 crossdb-cli CrossDB command line tool examples/ CrossDB example code"},{"location":"docs/get-started/#linuxfreebsd","title":"Linux/FreeBSD","text":""},{"location":"docs/get-started/#run-examples","title":"Run Examples","text":"
cd examples\n\n./build.sh example.c\n./example.bin\n\n./build.sh tutorial.c\n./tutorial.bin\n
"},{"location":"docs/get-started/#build-in-your-project","title":"Build in your project","text":"

  • You can check examples/build.sh to build the library in your project folder.

  • Yo can also install CrossDB globally and use it as common library.

    1. Install libcrossdb.so to /usr/lib

    #include \"crossdb.h\"\n// your code\n

    2. Build this way: gcc my.c -lcrossdb -pthread -ldl

"},{"location":"docs/get-started/#windows","title":"Windows","text":""},{"location":"docs/get-started/#run-examples-with-msvc","title":"Run Examples with MSVC","text":"

Start Visual Studio command line from menu x64 Native Tools Command Prompt for VS 20xx

Enter CrossDB package folder

cd examples\n\nbuild example.c\nexample.exe\n\nbuild tutorial.c\ntutorial.exe\n
"},{"location":"docs/get-started/#run-examples-with-mingw64","title":"Run Examples with MINGW64","text":"
cd examples\n\n./build.sh example.c\n./example.bin\n\n./build.sh tutorial.c\n./tutorial.bin\n

Note

You can run in git bash or MSYS2 MINGW64 If you only have MINGW64, you an run in commandline

gcc -o example.exe -Wall -O2 example.c -I.. crossdb.dll

"},{"location":"docs/get-started/#build-in-your-project_1","title":"Build in your project","text":"
  • For MINGW64, you can just use crossdb.h and crossdb.dll to compile.
  • For Visual Studio, You can add crossdb.h crossdb.lib and crossdb.dll to your project.
"},{"location":"docs/get-started/#macos","title":"MacOS","text":""},{"location":"docs/get-started/#run-examples_1","title":"Run Examples","text":"
cd examples\n\n./build.sh example.c\n./example.bin\n\n./build.sh tutorial.c\n./tutorial.bin\n
"},{"location":"docs/get-started/#build-in-your-project_2","title":"Build in your project","text":"

  • You can check examples/build.sh to build the library in your project folder.

  • Yo can also install CrossDB globally and use it as common library.

    1. Install libcrossdb.dylib to /usr/local/lib

    #include \"crossdb.h\"\n// your code\n

    2. Build this way: clang my.c -lcrossdb

"},{"location":"docs/get-started/#use-crossdb-cli","title":"Use CrossDB CLI","text":"
../crossdb-cli db_data\nCrossDB> insert into route set prefix=1.1.1.1 mask=24 nexthop=1.1.1.254 metric=5 intf='eth1'\nCrossDB> select * from route\nID  prefix   mask  nexthop    metric  intf  birth                flags\n==  =======  ====  =========  ======  ====  ===================  =====\n1   1.1.1.1  24    1.1.1.254  5       eth1  1970-01-01T08:00:00  0x0\nQualified Rows: 1       Select Rows: 1    Use time 2us QPS 500000\n

More commands can be found here

"},{"location":"docs/introduction/","title":"Introduction","text":"

CrossDB is a powerful high-performance embedded RDBMS database. It's developed to improve development efficiency for embedded application or embedded system development.

"},{"location":"docs/introduction/#features","title":"Features","text":"
  • Support Multiple OS Platform: Linux/Windows/MacOS/FreeBSD etc
  • Support X86/ARM/PPC/MIPS etc CPU
  • Support OnDisk/In-memory/RamDisk Storage
  • Support Primary Key and multiple Secondary Index
  • Support HASH and RBTREE Index
  • Support Multi-columns Index
  • Support Exact Match, Leftmost Match, Range Match
  • Support C/C++ Struct/Union ORM APIs
  • Support Struct Scheme Auto upgrade/downgrade
  • Support almost all C data types
  • Support rich DEC/HEX/BIN/Bitmap/MAC/IPv4/IPv6/Timestamp formats
  • Support Standard Transaction (begin/commit/rollback)
  • Support WAL for OnDisk storage
  • Dynamic Growing/Shrinking Table
  • Support Multiple Threads and Multiple Processes Access
  • Support Table level read/write lock
  • Super High Performance
  • Very Robust: Process Restart, Process Crash, Power Cycle
  • Very Simple: Simple header and library file
  • Zero Config: no complex config, real out-of-the-box
"},{"location":"docs/introduction/#use-cases","title":"Use Cases","text":"
  • You can use CrossDB RDBMS to Manage Program Runtime Data efficiently.
  • You can use CrossDB transaction to do persistency storage on Disk/Flash with ACID feature.
  • YOu can use CrossDB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CrossDB RDBMS to refactor your code conveniently.
  • You can use CrossDB Index to optimize performance without changing your code.
"},{"location":"docs/api/api/","title":"API List","text":""},{"location":"docs/api/api/#database-definition-apis","title":"Database Definition APIs","text":"API Descritpion cross_ret cross_dbCreate (cross_db_h *phDb, const char *dbName, uint32_t flags) Create or Open Database cross_ret cross_dbClose (cross_db_h hDb, uint32_t flags) Close Database cross_ret cross_dbDrop (cross_db_h hDb, uint32_t flags) Drop Database cross_ret cross_dbTblCreate (cross_db_h hDb, cross_tbl_h *phTbl, const char *tblName, const cross_field_t *pFields, const char *priKey, uint32_t flags) Create or Open Table cross_ret cross_dbTblDrop (cross_tbl_h hTbl, uint32_t flags) Drop Table cross_ret cross_dbIdxCreate (cross_tbl_h hTbl, const char *idxName, const char *fldsStr, uint32_t flags) Create Index on Table cross_ret cross_dbIdxDrop (cross_tbl_h hTbl, const char *idxName, uint32_t flags) Drop Table Index"},{"location":"docs/api/api/#database-manipulation-apis","title":"Database Manipulation APIs","text":"API Descritpion cross_ret cross_dbInsertRow (cross_tbl_h hTbl, void *pRow, uint32_t flags) Intert row into Table cross_ret cross_dbReplaceRow (cross_tbl_h hTbl, void *pInRow, uint32_t flags) Insert/update row cross_ret cross_dbGetRowByPK (cross_tbl_h hTbl, const void *pInRow, void *pOutRow, uint32_t flags) Get one row by Primary Key cross_ret cross_dbUpdRowByPK (cross_tbl_h hTbl, const void *pInRow, const void *pUpdFlds, void *pUpdRow, uint32_t flags) Update row by Primary Key cross_ret cross_dbDelRowByPK (cross_tbl_h hTbl, void *pInRow, uint32_t flags) Delete row by Primary Key cross_ret cross_dbGetOneRow (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, void *pOutRow, uint32_t flags) Get one row by match fields cross_rowid cross_dbUpdateRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, const void *pUpdFlds, const void *pUpdRow, uint32_t flags) Update rows by match fields cross_rowid cross_dbDeleteRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags) Delete rows by match fields cross_rowid cross_dbGetRowsCount (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags) Get rows count by match fields cross_rowid cross_dbQueryRows (cross_tbl_h hTbl, cross_cursor_h *phCursor, const void *pMatFlds, const void *pMatRow, uint32_t flags) Query rows into cursor cross_ret cross_cursorGetNextRow (cross_cursor_h hCursor, void *pOutRow, uint32_t flags) Get next row in cursor cross_rowid cross_cursorGetCount (cross_cursor_h hCursor, uint32_t flags) Get cursor rows count cross_ret cross_cursorClose (cross_cursor_h hCursor, uint32_t flags) Close cursor cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags) Begin Transaction cross_ret cross_dbTransCommit (cross_db_h hDb, uint32_t flags) Commit Transaction cross_ret cross_dbTransRollback (cross_db_h hDb, uint32_t flags) Rollback Transaction"},{"location":"docs/api/api/#misc-apis","title":"Misc APIs","text":"API Descritpion const char* cross_errMsg (cross_db_h hDb, uint32_t flags) Return error code message cross_ret cross_fieldsCreate (cross_tbl_h hTbl, cross_fields_h *phFlds, const char *FldsStr, uint32_t flags) Create fields handle void cross_fieldsFree (cross_fields_h hFlds) Free fields handle cross_ret cross_matchCreate (cross_tbl_h hTbl, cross_match_h *phMatch, const char *MatchStr, uint32_t flags) Create match handle void cross_matchFree (cross_match_h hMatch) Free match handle"},{"location":"docs/api/ddl/","title":"Database Definition APIs","text":""},{"location":"docs/api/ddl/#database","title":"Database","text":""},{"location":"docs/api/ddl/#cross_dbCreate","title":"cross_dbCreate","text":"

Create or Open Database 

cross_ret cross_dbCreate (cross_db_h *phDb, const char *dbName, uint32_t flags);\n
Parameters

Arg Type Descritpion phDb out DB Handle dbName in DB Name(can with Path) flags [in] See following tables Flag Descritpion CROSS_ONDISK DB is on persistent disk, survive with power cycle (Default) CROSS_RAMDISK DB is on ramdisk/tmpfs/ramfs, survive with process restart, lose after power cycle CROSS_INMEM DB is in memory, survie when process is runnig, lose after process terminates CROSS_EXCLUSIVE DB is used exclusively by single process (Default) CROSS_SHARED DB is shared by multiple processes CROSS_AUTOLOCK DB will do lock automatically (Default) CROSS_OPEN Open DB (don't create if not exist)

Returns

  • CROSS_OK: The database was created/opened successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/ddl/#cross_dbClose","title":"cross_dbClose","text":"

Close Database 

cross_ret cross_dbClose (cross_db_h hDb, uint32_t flags);\n
Parameters

Arg Type Descritpion hDb in DB Handle flags [in] Not used

Returns

  • CROSS_OK: The database was closed successfully
  • Other: Decode with cross_errMsg

Warning

DB handle can't be used after close.

"},{"location":"docs/api/ddl/#cross_dbDrop","title":"cross_dbDrop","text":"

Drop Database 

cross_ret cross_dbDrop (cross_db_h hDb, uint32_t flags);\n
Parameters

Arg Type Descritpion hDb in DB Handle flags [in] Not used

Returns

  • CROSS_OK: The database was dropped successfully
  • Other: Decode with cross_errMsg

Warning

All tables and indexes will be removed. DB handle can't be used after drop.

"},{"location":"docs/api/ddl/#table","title":"Table","text":""},{"location":"docs/api/ddl/#cross_dbTblCreate","title":"cross_dbTblCreate","text":"

Create or Open Table 

cross_ret cross_dbTblCreate (cross_db_h hDb, cross_tbl_h *phTbl, const char *tblName,\nconst cross_field_t *pFields, const char *priKey, uint32_t flags);\n

Parameters

Arg Type Descritpion hDb in DB Handle phDb out Table Handle tblName in Table Name pFields [in] Table Schema Fields priKey [in] Primary Key cloumn list seperated by , flags [in] See following tables Flag Descritpion CROSS_RBTREE Primary Key is RBTREE CROSS_OPEN Open DB (don't create if not exist)

Returns

  • CROSS_OK: The table was created/opened successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/ddl/#cross_dbTblDrop","title":"cross_dbTblDrop","text":"

Drop Table 

cross_ret cross_dbTblDrop (cross_tbl_h hTbl, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle flags [in] Not used

Returns

  • CROSS_OK: The table was dropped successfully
  • Other: Decode with cross_errMsg

Warning

Table handle can't be used after drop.

"},{"location":"docs/api/ddl/#index","title":"Index","text":""},{"location":"docs/api/ddl/#cross_dbIdxCreate","title":"cross_dbIdxCreate","text":"

Create Index on Table 

cross_ret cross_dbIdxCreate (cross_tbl_h hTbl, const char *idxName, const char *fldsStr, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle idxName in index name(table scope) fldsStr in cloumn list seperated by , flags [in] See following tables Flag Descritpion CROSS_HASH index type HASH (default) CROSS_RBTREE index type RBTREE CROSS_UNIQUE index is Unique

Returns

  • CROSS_OK: The index was created successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/ddl/#cross_dbIdxDrop","title":"cross_dbIdxDrop","text":"

Drop Table Index 

cross_ret cross_dbIdxDrop (cross_tbl_h hTbl, const char *idxName, uint32_t flags);\n
Parameters

Arg Type Descritpion hTbl in Table Handle idxName in index name(table scope) flags [in] Not used

Returns

  • CROSS_OK: The index was dropped successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/dml/","title":"Database Manipulation APIs","text":""},{"location":"docs/api/dml/#single-row-manipulation","title":"Single Row Manipulation","text":""},{"location":"docs/api/dml/#cross_dbInsertRow","title":"cross_dbInsertRow","text":"

Intert row to Table 

cross_ret cross_dbInsertRow (cross_tbl_h hTbl, void *pRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pRow in Row buffer to insert pValFlds [in] Not used flags [in] Not used

Returns

  • CROSS_OK: The row was inserted successfully
  • Other: Decode with cross_errMsg

Description

  • If table has Primary Key or unique index, it must be valid and unique.
"},{"location":"docs/api/dml/#cross_dbReplaceRow","title":"cross_dbReplaceRow","text":"

Insert row to Table if not exist else replace row 

cross_ret cross_dbReplaceRow (cross_tbl_h hTbl, void *pInRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pRow in Row buffer to insert/update pNullFlds in Not used flags [in] Not used

Returns

  • CROSS_OK: The row was inserted/updated successfully
  • Other: Decode with cross_errMsg

Description

  • If table has Primary Key or unique index, it must be valid and unique.
"},{"location":"docs/api/dml/#cross_dbGetRowByPK","title":"cross_dbGetRowByPK","text":"

Get one row by Primary Key 

cross_ret cross_dbGetRowByPK (cross_tbl_h hTbl, const void *pInRow, void *pOutRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pInRow in Row buffer with Primary Key pOutRow [out] Row buffer to return DB row flags [in] Not used

Returns

  • CROSS_OK: Get row successfully
  • CROSS_E_NOTFOUND: The row is not found
  • Other: Decode with cross_errMsg

Description

  • pOutRow = NULL can be used to check if row exists by Primary Key
  • pOutRow can = pInRow
"},{"location":"docs/api/dml/#cross_dbUpdRowByPK","title":"cross_dbUpdRowByPK","text":"

Update row by Primary Key 

cross_ret cross_dbUpdRowByPK (cross_tbl_h hTbl, const void *pInRow, const void *pUpdFlds, void *pUpdRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pInRow in Row buffer with Primary Key pUpdFlds [in] Fields to update pUpdRow in Row buffer to upate flags [in] Not used

Returns

  • CROSS_OK: Update row successfully
  • CROSS_E_NOTFOUND: The row is not found
  • Other: Decode with cross_errMsg

Description

  • pUpdFlds can be cloumn list seperated by ,
  • pUpdFlds can be handle created by cross_fieldsCreate
  • pUpdFlds = NULL will update whole DB row
  • pUpdRow can = pInRow
"},{"location":"docs/api/dml/#cross_dbDelRowByPK","title":"cross_dbDelRowByPK","text":"

Delete row by Primary Key 

cross_ret cross_dbDelRowByPK (cross_tbl_h hTbl, void *pInRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pInRow in Row buffer with Primary Key flags [in] Not used

Returns

  • CROSS_OK: Delete row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/dml/#cross_dbGetOneRow","title":"cross_dbGetOneRow","text":"

Get one row by match fields 

cross_ret cross_dbGetOneRow (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, void *pOutRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pMatFlds in Fields to match pMatRow in Row buffer with match row pOutRow [out] Row buffer to return DB row flags [in] Not used

Returns

  • CROSS_OK: Get row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg

Description

  • pMatFlds can be cloumn list seperated by ,, they are AND together.
    • fld: equal to match dbRow.fld = pMatRow.fld.
    • fld!=: unequal match dbRow.fld != pMatRow.fld.
    • fld>: greater than match dbRow.fld > pMatRow.fld.
    • fld>=: greater than or equal to match dbRow.fld >= pMatRow.fld.
    • fld<: less match dbRow.fld < pMatRow.fld.
    • fld<=: leas than or equal to match dbRow.fld < pMatRow.fld.
  • pMatFlds can be handle created by cross_matchCreate
  • pOutRow = NULL can be used to check if row exists
"},{"location":"docs/api/dml/#multiple-rows-manipulation","title":"Multiple Rows Manipulation","text":""},{"location":"docs/api/dml/#cross_dbUpdateRows","title":"cross_dbUpdateRows","text":"

Update rows by match fields 

cross_rowid cross_dbUpdateRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, const void *pUpdFlds, const void *pUpdRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pMatFlds in Fields to match pMatRow in Row buffer with match row pUpdFlds [in] Upate specified fields pUpdRow in Row buffer to update flags [in] Not used

Returns

  • >=0: Rows count updated successfully
  • Other: Decode with cross_errMsg

Description

  • pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow
  • pMatFlds can be handle created by cross_matchCreate
  • pUpdFlds can be cloumn list seperated by ,,
  • pUpdFlds can be handle created by cross_fieldsCreate
  • pUpdFlds = NULL updates whole row
"},{"location":"docs/api/dml/#cross_dbDeleteRows","title":"cross_dbDeleteRows","text":"

Delete rows by match fields 

cross_rowid cross_dbDeleteRows (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pMatFlds in Fields to match pMatRow in Row buffer with match row flags [in] Not used

Returns

  • >=0: Rows count deleted successfully
  • Other: Decode with cross_errMsg

Description - pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow - pMatFlds can be handle created by cross_matchCreate

"},{"location":"docs/api/dml/#cross_dbGetRowsCount","title":"cross_dbGetRowsCount","text":"

Get rows count by match fields 

cross_rowid cross_dbGetRowsCount (cross_tbl_h hTbl, const void *pMatFlds, const void *pMatRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle pMatFlds in Fields to match pMatRow in Row buffer with match row flags [in] Not used

Returns

  • >=0: Rows count
  • Other: Decode with cross_errMsg

Description - pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow - pMatFlds can be handle created by cross_matchCreate

"},{"location":"docs/api/dml/#cursor","title":"Cursor","text":""},{"location":"docs/api/dml/#cross_dbQueryRows","title":"cross_dbQueryRows","text":"

Query rows into cursor 

cross_rowid cross_dbQueryRows (cross_tbl_h hTbl, cross_cursor_h *phCursor, const void *pMatFlds, const void *pMatRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle phCursor io|out Return Cursor handle or pMatFlds in Fields to match pMatRow in Row buffer with match row flags [in] CROSS_REUSE means phCursor points to a valid cursor handle and it'll be reused

Returns

  • >=0: Rows count
  • Other: Decode with cross_errMsg

Description

  • pMatFlds can be cloumn list seperated by ,, they are AND together. Refer cross_dbGetOneRow
  • pMatFlds can be handle created by cross_matchCreate
"},{"location":"docs/api/dml/#cross_cursorGetNextRow","title":"cross_cursorGetNextRow","text":"

Get next row in cursor 

cross_ret cross_cursorGetNextRow (cross_cursor_h hCursor, void *pOutRow, uint32_t flags);\n

Parameters

Arg Type Descritpion hCursor in Cursor Handle pOutRow [out] Row buffer to return DB row flags [in] Not used

Returns

  • CROSS_OK: Get next row successfully
  • CROSS_E_NOTFOUND: No row is found
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/dml/#cross_cursorgetcount","title":"cross_cursorGetCount","text":"

Get cursor rows count 

cross_rowid cross_cursorGetCount (cross_cursor_h hCursor, uint32_t flags);\n

Parameters

Arg Type Descritpion hCursor in Cursor Handle flags [in] Not used

Returns

  • >=0: Rows count
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/dml/#cross_cursorClose","title":"cross_cursorClose","text":"

Close cursor 

cross_ret cross_cursorClose (cross_cursor_h hCursor, uint32_t flags);\n

Parameters

Arg Type Descritpion hCursor in Cursor Handle flags [in] Not used

Returns

  • >=0: Rows count
  • Other: Decode with cross_errMsg

Warning

Cursor handle can't be used after close.

"},{"location":"docs/api/dml/#transaction","title":"Transaction","text":""},{"location":"docs/api/dml/#cross_dbTransBegin","title":"cross_dbTransBegin","text":"

Begin Transaction 

cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags);\n

Parameters

Arg Type Descritpion hDb in DB Handle flags [in] Not used

Returns

  • CROSS_OK: Begin transaction successfully
  • Other: Decode with cross_errMsg

Note

If there's onging transaction, it'll be commited first.

"},{"location":"docs/api/dml/#cross_dbTransCommit","title":"cross_dbTransCommit","text":"

Commit Transaction 

cross_ret cross_dbTransCommit (cross_db_h hDb, uint32_t flags);\n

Parameters

Arg Type Descritpion hDb in DB Handle flags [in] Not used

Returns

  • CROSS_OK: Commit transaction successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/dml/#cross_dbTransRollback","title":"cross_dbTransRollback","text":"

Rollback Transaction 

cross_ret cross_dbTransRollback (cross_db_h hDb, uint32_t flags);\n

Parameters

Arg Type Descritpion hDb in DB Handle flags [in] Not used

Returns

  • CROSS_OK: Rollback transaction successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/misc/","title":"Misc APIs","text":""},{"location":"docs/api/misc/#cross_errMsg","title":"cross_errMsg","text":"

Return error code message 

const char*\ncross_errMsg (cross_ret ret);\n

"},{"location":"docs/api/misc/#cross_fieldsCreate","title":"cross_fieldsCreate","text":"

Create fields handle 

cross_ret   cross_fieldsCreate (cross_tbl_h hTbl, cross_fields_h *phFlds, const char *FldsStr, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle phFlds out Fields Handle FldsStr in Cloumn list seperated by , flags [in] Not used

Returns

  • CROSS_OK: Create handle successfully
  • Other: Decode with cross_errMsg
"},{"location":"docs/api/misc/#cross_fieldsFree","title":"cross_fieldsFree","text":"

Free fields handle 

void        cross_fieldsFree (cross_fields_h hFlds);\n

"},{"location":"docs/api/misc/#cross_matchCreate","title":"cross_matchCreate","text":"

Create match handle 

cross_ret   cross_matchCreate (cross_tbl_h hTbl, cross_match_h *phMatch, const char *MatchStr, uint32_t flags);\n

Parameters

Arg Type Descritpion hTbl in Table Handle phMatch out Match Handle MatchStr in Match list seperated by , flags [in] Not used

Returns

  • CROSS_OK: Create handle successfully
  • Other: Decode with cross_errMsg

Description

  • MatchStr cloumn list seperated by ,, they are AND together. Refer dml.md#cross_dbGetOneRow
    • fld: equal to match dbRow.fld = pMatRow.fld.
    • fld!=: unequal match dbRow.fld != pMatRow.fld.
    • fld>: greater than match dbRow.fld > pMatRow.fld.
    • fld>=: greater than or equal to match dbRow.fld >= pMatRow.fld.
    • fld<: less match dbRow.fld < pMatRow.fld.
    • fld<=: leas than or equal to match dbRow.fld < pMatRow.fld.
"},{"location":"docs/api/misc/#cross_matchFree","title":"cross_matchFree","text":"

Free match handle 

void        cross_matchFree (cross_match_h hMatch);\n

"},{"location":"docs/crossdb/database/","title":"Database","text":"

CrosDB Database is a collection of tables.

Steps to define a database

  • Decide where to store the database: on disk, on ramdisk or in memory.
  • Decide the database is accessed by single process or multiple proceses.
  • Use cross_dbCreate to create or open database.
  • Use Schema to define table scheam.
  • Decide Table Primary Key: fields list, index type.
  • Use cross_dbTblCreate to create or open table with schema defined above and provide the primary key.
  • Decide how many secondary indexes: fields list, index type, whether unique.
  • Use cross_dbIdxCreate to create table secondary index.
"},{"location":"docs/crossdb/database/#storage-mode","title":"Storage Mode","text":"

On-Disk Database

  • Database is on persistent disk, survive with power cycle.
  • If you have data persistency requirements, please use this type. The performance is ver high.
  • You need use transaction to guarantee data integrity even power cycle happened.

RamDisk Database

  • Database is on ramdisk tmpfs ramfs(CROSS_RAMDISK), survive with process restart, lose after power cycle.
  • This is designed for process runtime databse and the transaction performance is higher than On-Disk database.
  • The performance is almost the same with In-Memory database.
  • For Linux embedded system, this is the prefered database as the database can be viewed and stayed there even process crashed.

In-Memory Database

  • Database is in memory(CROSS_INMEM), survie when process is runnig, lose after process terminates.
  • This is not recommented, but if you don't want the process runtime database visible, it's the solution.
"},{"location":"docs/crossdb/database/#access-mode","title":"Access Mode","text":"

Exclusive

  • Database is accessed exclusively by single process with best performance.

Shared

  • Databse is accessd by multiple process(CROSS_SHARED), then performance will be low as file lock is used to access DB which can recover lock when process crashed.

Note

Database is accessed exclusively by default.

"},{"location":"docs/crossdb/database/#guide","title":"Guide","text":"
  • Database is stored on disk by default.
  • If path(absoulte or relative) is not provided, DB is stored in current folder.
"},{"location":"docs/crossdb/database/#example","title":"Example","text":"
#define CHECK(ret,str)  \\\n    if (ret < 0) {  printf (str\": %s\\n\", cross_errMsg(ret)); return -1; }\ncross_ret       ret;\ncross_db_h      hDb;\n// Create on disk database in current folder for single process access\nret = cross_dbCreate (&hDb, \"ondiskdb\", 0);\nCHECK (ret, \"Failed to create database\");\n// Create on disk database for multiple process access\nret = cross_dbCreate (&hDb, \"cdb_data/shareddb\", CROSS_SHARED);\nCHECK (ret, \"Failed to create database\");\n// Create ramdisk database for single process access\nret = cross_dbCreate (&hDb, \"/tmp/cdb_data/mydb\", CROSS_RAMDISK);\nCHECK (ret, \"Failed to create database\");\n// Create in memory database\nret = cross_dbCreate (&hDb, \"imdb\", CROSS_INMEM);\nCHECK (ret, \"Failed to create database\");\n
"},{"location":"docs/crossdb/dml/","title":"Data Manipulation Guide","text":""},{"location":"docs/crossdb/dml/#define-database","title":"Define Database","text":""},{"location":"docs/crossdb/dml/#handle-and-macros","title":"Handle and Macros","text":"
cross_ret       ret;\nroute_t         route;  cross_rowid     count;\ncross_db_h      hDb = NULL;\ncross_tbl_h     hRtTbl = NULL;\ncross_fields_h  hFlagsFlds = NULL;\ncross_match_h   hNexthopMat = NULL, hNexthopNeMat = NULL;\n#define CHECK(ret,str)      if (ret < 0) {  printf (str\": %s\\n\", cross_errMsg(ret)); return -1; }\n#define EXPECT(val,exp,str) if (val != exp) {   printf (\"%s: %d != %d\\n\", str, val, exp); return -1; } \\\n                            else  { printf (\"%s: %d\\n\", str, val); }\n#define IP4ADDR(a,b,c,d)        (uint32_t)((a)<<24|(b)<<16|(c)<<8|(d))\n#define IP4STR(ip)              ip>>24,(ip>>16)&0xff,(ip>>8)&0xff,ip&0xff\n#define DUMP_ROUTE(str, route)  printf (str\"%d.%d.%d.%d/%d->%d.%d.%d.%d intf: %s metric: %d flags: 0x%x\\n\",     \\\n                        IP4STR(route.prefix), route.mask, IP4STR(route.nexthop), route.intf, route.metric, route.flags);\n
"},{"location":"docs/crossdb/dml/#define-schema","title":"Define Schema","text":"
typedef struct {\nuint32_t            prefix;\nuint8_t             mask;\nuint32_t            nexthop;\nuint8_t             metric;\nchar                intf[16];\ntime_t              birth;\nuint32_t            flags;\n} route_t;\n#undef  CROSS_STRUCT_NAME\n#define CROSS_STRUCT_NAME   route_t\ncross_field_t   route_schema[] = {\nCROSS_FIELD (prefix,    UINT,   IPv4, 0),\nCROSS_FIELD (mask,      UINT,   DFT,  0),\nCROSS_FIELD (nexthop,   UINT,   IPv4, 0),\nCROSS_FIELD (metric,    UINT,   DFT,  0),\nCROSS_FIELD (intf,      CHAR,   DFT,  0),\nCROSS_FIELD (birth,     UINT,   TS,   0),\nCROSS_FIELD (flags,     UINT,   HEX,  0),\nCROSS_END (route_t)\n};\n
"},{"location":"docs/crossdb/dml/#create-db","title":"Create DB","text":"
printf (\"\\n============ Create DB & Table ============\\n\");\nprintf (\"  Create database : tutorial\\n\");\nret = cross_dbCreate (&hDb, \"db_data/tutorial\", 0);\nCHECK (ret, \"Failed to create DB: tutorial\");\nprintf (\"  Create table: route (PrimaryKey: prefix,mask)\\n\");\nret = cross_dbTblCreate (hDb, &hRtTbl, \"route\", route_schema, \"prefix,mask\", 0);\nCHECK (ret, \"Failed to create table: route\");\nprintf (\"  Create index on nexthop: idx_nexthop\\n\");\nret = cross_dbIdxCreate (hRtTbl, \"idx_nexthop\", \"nexthop\", 0);\nCHECK (ret, \"Failed to create index: idx_nexthop\");\n// Delete all rows\ncross_dbDeleteRows (hRtTbl, NULL, NULL, 0);\nret = cross_fieldsCreate (hRtTbl, &hFlagsFlds, \"flags\", 0);\nCHECK (ret, \"Failed to create fields: flags\");\nret = cross_matchCreate (hRtTbl, &hNexthopMat, \"nexthop\", 0);\nCHECK (ret, \"Failed to create fields: nexthop\");\nret = cross_matchCreate (hRtTbl, &hNexthopNeMat, \"nexthop!=\", 0);\nCHECK (ret, \"Failed to create fields: nexthop!=\");\n
"},{"location":"docs/crossdb/dml/#manipulation-database","title":"Manipulation Database","text":""},{"location":"docs/crossdb/dml/#insert-rows","title":"Insert Rows","text":"
printf (\"\\n============ Insert Rows ============\\n\");\nprintf (\"  Insert route 192.168.1.0/24->192.168.1.254\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(192,168,1,254);\nroute.metric    = 1;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth1\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert route 192.168.1.0/24\");\nprintf (\"  Insert route 10.1.1.0/24->10.1.2.254\\n\");\nroute.prefix    = IP4ADDR(10,1,1,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(10,1,2,254);\nroute.metric    = 2;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth2\");\nroute.birth     = time (NULL);\nret = cross_dbInsertRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to insert route 10.1.1.0/24\");\nprintf (\"  Use Replace to insert route 10.1.2.0/24->10.1.2.254\\n\");\nroute.prefix    = IP4ADDR(10,1,2,0);\nroute.mask      = 24;   route.nexthop   = IP4ADDR(10,1,2,254);\nroute.metric    = 1;\nroute.flags     = 0;\nstrcpy (route.intf, \"eth2\");\nroute.birth     = time (NULL);\nret = cross_dbReplaceRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to replace route 10.1.2.0/24\");\n
"},{"location":"docs/crossdb/dml/#query-rows","title":"Query Rows","text":"
printf (\"\\n============ Query Rows ============\\n\");\n// Get all rows count\ncount = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);\nEXPECT (count, 3, \"  Total routes\");\n// Get rows count where nexthop=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {\ncount = cross_dbGetRowsCount (hRtTbl, \"nexthop\", &route, 0);\n} else {\ncount = cross_dbGetRowsCount (hRtTbl, hNexthopMat, &route, 0);\n}\nEXPECT (count, 2, \"  Nexthop=10.1.2.254 routes\");\n// Get single route 192.168.1.0/24 by PrimaryKey\nmemset (&route, 0, sizeof(route));\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0); CHECK (ret, \"Failed to get route 192.168.1.0/24 by PrimaryKey\");\nDUMP_ROUTE (\"  Get single route: \", route);\n// Get one row where nexthop=10.1.2.254\nmemset (&route, 0, sizeof(route));  route.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {\nret = cross_dbGetOneRow (hRtTbl, \"nexthop\", &route, &route, 0);\n} else {\nret = cross_dbGetOneRow (hRtTbl, hNexthopMat, &route, &route, 0);\n}\nCHECK (ret, \"Failed to get one route where nexthop=10.1.2.254\");\nDUMP_ROUTE (\"  Get one route where nexthop=10.1.2.254: \", route);\n// Get one row where nexthop!=10.1.2.254\nmemset (&route, 0, sizeof(route));  route.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {\nret = cross_dbGetOneRow (hRtTbl, \"nexthop!=\", &route, &route, 0);\n} else {\nret = cross_dbGetOneRow (hRtTbl, hNexthopNeMat, &route, &route, 0);\n}\nCHECK (ret, \"Failed to get one route where nexthop!=10.1.2.254\");\nDUMP_ROUTE (\"  Get one route where nexthop!=10.1.2.254: \", route);\n
"},{"location":"docs/crossdb/dml/#update-rows","title":"Update Rows","text":"
printf (\"\\n============ Update Rows ============\\n\");\nprintf (\"  Update single route 192.168.1.0/24 by Primary Key: set flags ->1\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 1;\nif (!optimize) {\nret = cross_dbUpdRowByPK (hRtTbl, &route, \"flags\", &route, 0); } else {\nret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); }\nCHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\n// verify\nroute.flags     = 0;\nret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nEXPECT (route.flags, 1, \"    Update flags\");\nDUMP_ROUTE (\"    Get single route: \", route);\nprintf (\"  Update single route 192.168.1.0/24 by Primary Key: update whole row\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nCHECK (ret, \"Failed to get route 192.168.1.0/24 by PrimaryKey\");\nroute.flags     = 2;\nroute.metric    = 2;\nret = cross_dbUpdRowByPK (hRtTbl, &route, NULL, &route, 0); CHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\n// verify\nroute.flags     = 0;\nret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nEXPECT (route.flags, 2, \"    Update flags\");\nEXPECT (route.metric, 2, \"    Update metric\");\nDUMP_ROUTE (\"    Get single route: \", route);\nprintf (\"  Use Replace to Update single route 192.168.1.0/24 by Primary Key: set flags ->3\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 3;\nret = cross_dbReplaceRow (hRtTbl, &route, 0); CHECK (ret, \"Failed to replace route 192.168.1.0/24\");\n// verify\nroute.flags     = 0;\nret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nEXPECT (route.flags, 3, \"    Update flags\");\nDUMP_ROUTE (\"    Get single route: \", route);\n// Update routes where nexthop=10.1.2.254: set flags 0->4\nroute.nexthop   = IP4ADDR(10,1,2,254);\nroute.flags     = 4;\nif (!optimize) {\ncount = cross_dbUpdateRows (hRtTbl, \"nexthop\", &route, \"flags\", &route, 0);\n} else {\ncount = cross_dbUpdateRows (hRtTbl, hNexthopMat, &route, hFlagsFlds, &route, 0);\n}\nEXPECT (count, 2, \"  Update nexthop=10.1.2.254 set flags=4 rows\");\n// Update All routes set flags 0->5\nroute.flags     = 5;\nif (!optimize) {\ncount = cross_dbUpdateRows (hRtTbl, NULL, NULL, \"flags\", &route, 0);\n} else {\ncount = cross_dbUpdateRows (hRtTbl, NULL, NULL, hFlagsFlds, &route, 0);\n}\nEXPECT (count, 3, \"  Update all routes set flags=4 rows\");\n
"},{"location":"docs/crossdb/dml/#use-cursor","title":"Use Cursor","text":"
printf (\"\\n============ Cursor Query ============\\n\");\n// Use cursor to get all routes\ncross_cursor_h hCursor;\ncount = cross_dbQueryRows (hRtTbl, &hCursor, NULL, &route, 0);\nEXPECT (count, 3, \"  Query All routes\");\nwhile (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {\nDUMP_ROUTE (\"    route: \", route);\n}\n// Reuse cursor to get routes where nexthop=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {\ncount = cross_dbQueryRows (hRtTbl, &hCursor, \"nexthop\", &route, CROSS_REUSE);\n} else {\ncount = cross_dbQueryRows (hRtTbl, &hCursor, hNexthopMat, &route, CROSS_REUSE);\n}   EXPECT (count, 2, \"  Query nexthop=10.1.2.254 routes\");\nwhile (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {\nDUMP_ROUTE (\"    route: \", route);\n}\n// Reuse cursor to get routes where nexthop!=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {    count = cross_dbQueryRows (hRtTbl, &hCursor, \"nexthop!=\", &route, CROSS_REUSE);\n} else {\ncount = cross_dbQueryRows (hRtTbl, &hCursor, hNexthopNeMat, &route, CROSS_REUSE);\n}\nEXPECT (count, 1, \"  Query nexthop!=10.1.2.254 routes\");\nwhile (CROSS_OK == cross_cursorGetNextRow (hCursor, &route, 0)) {\nDUMP_ROUTE (\"    route: \", route);\n}\ncross_cursorClose (hCursor, 0);\n
"},{"location":"docs/crossdb/dml/#use-transaction","title":"Use Transaction","text":"
printf (\"\\n============ Transaction ============\\n\");\nprintf (\"  Commit\\n\");\nret = cross_dbTransBegin (hDb, 0);\nCHECK (ret, \"Failed to begin transaction\");\nprintf (\"    Update single route 192.168.1.0/24 by Primary Key: set flags to 8\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 8;\nif (!optimize) {\nret = cross_dbUpdRowByPK (hRtTbl, &route, \"flags\", &route, 0); } else {\nret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); }\nCHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\nret = cross_dbTransCommit (hDb, 0);\nCHECK (ret, \"Failed to commit transaction\");\n// verify\nroute.flags     = 0;\nret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nEXPECT (route.flags, 8, \"    Update flags\");\nDUMP_ROUTE (\"    Get single route: \", route);\nprintf (\"  Rollback\\n\");\nret = cross_dbTransBegin (hDb, 0);\nCHECK (ret, \"Failed to begin transaction\");\nprintf (\"    Update single route 192.168.1.0/24 by Primary Key: set flags 0->9\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   route.flags     = 9;\nif (!optimize) {\nret = cross_dbUpdRowByPK (hRtTbl, &route, \"flags\", &route, 0); } else {\nret = cross_dbUpdRowByPK (hRtTbl, &route, hFlagsFlds, &route, 0); }\nCHECK (ret, \"Failed to update route 192.168.1.0/24 by Primary Key\");\nret = cross_dbTransRollback (hDb, 0);\nCHECK (ret, \"Failed to commit transaction\");\n// verify\nroute.flags     = 0;\nret = cross_dbGetRowByPK (hRtTbl, &route, &route, 0);\nEXPECT (route.flags, 8, \"    Rollback to orignal flags\");\nDUMP_ROUTE (\"    Get single route: \", route);\n
"},{"location":"docs/crossdb/dml/#delete-rows","title":"Delete Rows","text":"
printf (\"\\n============ Delete Rows ============\\n\");\nprintf (\"  Delete single route 192.168.1.0/24 by Primary Key\\n\");\nroute.prefix    = IP4ADDR(192,168,1,0);\nroute.mask      = 24;   ret = cross_dbDelRowByPK (hRtTbl, &route, 0); CHECK (ret, \"Failed to delete route 192.168.1.0/24 by Primary Key\");\n// verify\ncount = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);\nEXPECT (count, 2, \"    Total routes\");\n// Delete routes where nexthop=10.1.2.254\nroute.nexthop   = IP4ADDR(10,1,2,254);\nif (!optimize) {\ncount = cross_dbDeleteRows (hRtTbl, \"nexthop\", &route, 0);\n} else {\ncount = cross_dbDeleteRows (hRtTbl, hNexthopMat, &route, 0);\n}\nEXPECT (count, 2, \"  Delete nexthop=10.1.2.254 rows\");\n// verify\ncount = cross_dbGetRowsCount (hRtTbl, NULL, NULL, 0);\nEXPECT (count, 0, \"    Total routes\");\n
"},{"location":"docs/crossdb/dml/#close-db","title":"Close DB","text":"
printf (\"\\n============ Close DB ============\\n\");\ncross_dbClose (hDb, 0);\n
"},{"location":"docs/crossdb/indexes/","title":"Index","text":"

CrossDB Index is used to implement database Primary Key and accelerate query.

"},{"location":"docs/crossdb/indexes/#index-type","title":"Index Type","text":"

HASH Index

  • HASH index has highest O(1) performance. It's optimized a lot to achieve best performance.
  • HASH index can dynamically scale with table rows number to get high performance.
  • HASH index only supports exact match.

RBTREE Index

  • Use flag CROSS_RBTREE
  • RBTREE index is almost the same with popular RDBMS BTree, which is core index engine of RDBMS.
  • RBTREE index can support exact match, range match and leftmost prefix match with multiple-column.
"},{"location":"docs/crossdb/indexes/#guide","title":"Guide","text":"
  • Default index is HASH type.
  • If one query matches both HASH index and RBTREE index, then Hash index is selected in most cases.
  • You can create HASH index for high performance query and RBTREE index for normal performance queries.
  • Index is not free, it occupies space and all inexes may be updated during INSERT/UPDATE/DELET row.
  • If index is unique, please set CROSS_UNIQUE.
  • For unique index, either insert replace update will check row is unique.
  • You can do any query on any fields combination without index.
  • Create index only when the query calls frequency is high and requires performance.
"},{"location":"docs/crossdb/indexes/#example","title":"Example","text":"
#define CHECK(ret,str)  \\\n    if (ret < 0) {  printf (str\": %s\\n\", cross_errMsg(ret)); return -1; }\n// Create hash index\nret = cross_dbIdxCreate (hRtTbl, \"idx_nexthop\", \"nexthop\", 0);\nCHECK (ret, \"Failed to create index\");\n// Create rbtree index\nret = cross_dbIdxCreate (hRtTbl, \"idx_nexthop\", \"nexthop\", 0);\nCHECK (ret, \"Failed to create index\");\n// Create unique hash index\nret = cross_dbIdxCreate (hRtTbl, \"idx_route\", \"vrf,ipAddr\", CROSS_UNIQUE);\nCHECK (ret, \"Failed to create index\");\n// Create unique rbtree index\nret = cross_dbIdxCreate (hRtTbl, \"idx_route\", \"vrf,ipAddr\", CROSS_UNIQUE|CROSS_RBTREE);\nCHECK (ret, \"Failed to create index\");\n
"},{"location":"docs/crossdb/schema/","title":"Schema","text":""},{"location":"docs/crossdb/schema/#schema-definition","title":"Schema Definition","text":"

Schem is used to define CrossDB table and it's defined based on C struct with some CrossDB macro.

"},{"location":"docs/crossdb/schema/#basic-types","title":"Basic Types","text":"

CrossDB Type and C Type Map

CrossDB Type Description C Type INT integer char, short, int, long long, int8_t, int16_t, int32_t, int64_t UINT unsinged integer unsigned char, unsigned short, unsigned int, unsigned long long, uint8_t uint16_t, uint32_t, uint64_t FLOAT floating-point float, double CHAR charater string char [] BYTE byte array unsiged char [], uint8_t [], struct

CrossDB Format

CrossDB Type CrossDB Formt Description INT/UINT DFT default decimal FLOAT DFT default floating-point CHAR DFT default charater string BYTE DFT default hexadecimal string INT/UINT BOOL boolean UINT HEX hexadecimal INT/UINT TIMESTAMP/TS Timestamp: time_t, uin32_t, uint64_t BTYE MAC MAC addrss: uint8_t [6], struct ether_addr UINT IPv4 Host endian IPv4 address: uint32_t BYTE IPv4 Network endian IPv4 address: struct in_addr, uint32_t BYTE IPv6 Network endian IPv4 address: uint8_t [16], struct in_addr6

Example

typedef struct {\nchar            val_char;\nint64_t         val_int64;\nuint16_t        val_u16;\nuint8_t         val_u8;\nfloat           val_float;\ndouble          val_double;\nchar            val_str[16];\nuint8_t         val_byte[64];\nchar            val_bool;\nuint16_t        flags;\ntime_t          birth;\nunsigned char   mac[6];\nuint32_t        ipAddrHost;\nstruct in_addr  ipAddrNet;\nuint8_t         ip6Addr[16];\n} basic_types_t;\n

Set CROSS_STRUCT_NAME to the struct name basic_types_t for CROSS_FIELD to define the schema entry. CROSS_END is used to mark the end of schema definition.

#undef  CROSS_STRUCT_NAME\n#define CROSS_STRUCT_NAME   basic_types_t\ncross_field_t   basic_types_schema[] = {\nCROSS_FIELD (val_char,      INT,    DFT,  0),\nCROSS_FIELD (val_int64,     INT,    DFT,  0),\nCROSS_FIELD (val_u16,       UINT,   DFT,  0),\nCROSS_FIELD (val_u8,        UINT,   DFT,  0),\nCROSS_FIELD (val_float,     FLOAT,  DFT,  0),\nCROSS_FIELD (val_double,    FLOAT,  DFT,  0),\nCROSS_FIELD (val_str,       CHAR,   DFT,  0),\nCROSS_FIELD (val_byte,      BYTE,   DFT,  0),\nCROSS_FIELD (val_bool,      UINT,   BOOL, 0),\nCROSS_FIELD (flags,         UINT,   HEX,  0),\nCROSS_FIELD (birth,         UINT,   TS,   0),\nCROSS_FIELD (mac,           BYTE,   MAC,  0),\nCROSS_FIELD (ipAddrHost,    UINT,   IPv4, 0),\nCROSS_FIELD (ipAddrNet,     BYTE,   IPv4, 0),\nCROSS_FIELD (ip6Addr,       BYTEE,  IPv6, 0),\nCROSS_END (basic_types_t)\n};\n

To keep struct and schema in header file, you can define them together like following way.

Define macro ROUTE_SCHEMA() in header file 

// In header file\ntypedef struct {\nuint32_t            prefix;\nuint8_t             mask;\nuint32_t            nexthop;\nuint8_t             metric;\nchar                intf[16];\nuint32_t            birth;\nuint32_t            flags;\n} route_t;\n#dfine ROUTE_SCHEMA()   \\   \nCROSS_FIELD (prefix,    UINT,   IPv4, 0), \\\n    CROSS_FIELD (mask,      UINT,   DFT,  0), \\\n    CROSS_FIELD (nexthop,   UINT,   IPv4, 0), \\\n    CROSS_FIELD (metric,    UINT,   DFT,  0), \\\n    CROSS_FIELD (intf,      CHAR,   DFT,  0), \\\n    CROSS_FIELD (birth,     UINT,   TS,   0), \\\n    CROSS_FIELD (flags,     UINT,   HEX,  0), \\\n    CROSS_END (route_t)\n

Define schema in source file. CROSS_END can be define in route_schema also then you can reuse the macro ROUTE_SCHEMA() or combine them. 

    // In source file\n#undef  CROSS_STRUCT_NAME\n#define CROSS_STRUCT_NAME   route_t\ncross_field_t   route_schema[] = {\nROUTE_SCHEMA()\n}\n

"},{"location":"docs/crossdb/table/","title":"Table","text":"

CrosDB Table is a collection of rows(record or tuple) and columns.

  • Table schema is mapped from C Struct, so this is a kind of C Struct ORM(Object Relational Mapping).
  • The C struct is expanded into each single fields for composite data type like array, nested struct etc.
  • Each table can have only one Primary key.
  • Each table can hve many secondray indexes.
"},{"location":"docs/crossdb/table/#primary-keypk","title":"Primary Key(PK)","text":"
  • Each table can have only one Primray Key, which is used to identify the row uniquely.
  • Primay Key column list shouldn't change during the row lifecycle.
  • Primary Key is implemented with unique HASH index for high performance. You can change to RBTREE as well.
"},{"location":"docs/crossdb/table/#guide","title":"Guide","text":"
  • If table exists and pFields is different with table schema, table will be upgraded automatically.
  • If you want to know if table exists, you can use CROSS_OPEN to get handle first.
  • Primary Key is HASH type by default to achieve highest performance.
  • If you don't care about performance, you can set CROSS_RBTREE to create RBTREE type Primary Key.
  • If you need both exact match and range match for Primary Key, you can create another RBTREE index with same column list.
"},{"location":"docs/crossdb/table/#example","title":"Example","text":"
// Create table with PrimaryKey=\"prefix,mask\", HASH Type\nret = cross_dbTblCreate (hDb, &hRtTbl, \"route\", route_schema, \"prefix,mask\", 0);\nCHECK (ret, \"Failed to create route table\");\n// Create table with PrimaryKey=\"prefix,mask\", RBTREE Type\nret = cross_dbTblCreate (hDb, &hRtTbl, \"route\", route_schema, \"prefix,mask\", CROSS_RBTREE);\nCHECK (ret, \"Failed to create route table\");\n// Get table handle\nret = cross_dbTblCreate (hDb, &hRtTbl, \"route\", NULL, NULL, CROSS_OPEN);\nCHECK (ret, \"Failed to get route table\");\n
"},{"location":"docs/crossdb/transaction/","title":"Transaction","text":""},{"location":"docs/crossdb/transaction/#acid","title":"ACID","text":"

In the context of transaction processing, the acronym ACID refers to the four key properties of a transaction: atomicity, consistency, isolation, and durability.

Atomicity All changes to data are performed as if they are a single operation. That is, all the changes are performed, or none of them are. For example, in an application that transfers funds from one account to another, the atomicity property ensures that, if a debit is made successfully from one account, the corresponding credit is made to the other account.

Consistency Data is in a consistent state when a transaction starts and when it ends. For example, in an application that transfers funds from one account to another, the consistency property ensures that the total value of funds in both the accounts is the same at the start and end of each transaction.

Isolation The intermediate state of a transaction is invisible to other transactions. As a result, transactions that run concurrently appear to be serialized. For example, in an application that transfers funds from one account to another, the isolation property ensures that another transaction sees the transferred funds in one account or the other, but not in both, nor in neither.

Durability After a transaction successfully completes, changes to data persist and are not undone, even in the event of a system failure. For example, in an application that transfers funds from one account to another, the durability property ensures that the changes made to each account will not be reversed.

"},{"location":"docs/crossdb/transaction/#api","title":"API","text":"API Descritpion cross_ret cross_dbTransBegin (cross_db_h hDb, uint32_t flags) Begin Transaction cross_ret cross_dbTransCommit (cross_db_h hDb, uint32_t flags) Commit Transaction cross_ret cross_dbTransRollback (cross_db_h hDb, uint32_t flags) Rollback Transaction"},{"location":"docs/crossdb/transaction/#autocommit","title":"AutoCommit","text":"
  • For on-disk database, autocommit is enabled for single row and multiple rows DMLs by default.
  • For ramdisk and inmem database, autocommit is enabled for single row DMLs only by default. For multiple rows DMLs, if process crashed, some rows may be updated/deleted, some may not, so if you need strict ACID, you have to call transaction APIs manually. This is to improve performance and let user to control the behavior by need.
"},{"location":"docs/reference/crossbench/","title":"CrossDB Benchmark","text":"

CrossBench is an utility designed for evaluating performance of embedded databases. The goal of this project is to provide a standart and simple in use instrument for benchmarking, so any database developer or user can reference to or repeat obtained results.

  • Storage modes: On-Disk, In-Memory, On-Ramdisk
  • Benchmarking methods: insert, load, query(top 3 avg), update(top 5 avg), delete
  • Access Sequence: random, sequential
  • Thread: Single thread now, you can bind cpu core to do test
  • Supported databases: crossdb, sqlite3, lmdb, STL map, STL hashmap(unordered_map)
"},{"location":"docs/reference/crossbench/#usage","title":"Usage","text":"
./crossdb-bench.bin -h\nUsage: {count unit can be k or K (*1000)  m or M (*1000000)}\n  -h                        show this help\n  -s <db storage>           ramdisk | inmem | ondisk, default is ramdisk\n                            shortcut r: ramdisk  m: inmem    d: ondisk\n                            ondisk store db in current folder crossbench\n  -i <insert count>         default 1000000, do insert test, conflict with -l load\n  -r <row count>            open db which already has these rows\n  -q <query count>          default 1000000, do query test\n  -u <update count>         default 1000000, do update test\n  -d <delete count>         default 1000000, do delete test\n  -c <cpu core>             bind cpu core\n  -l <load count>           quick load data, conflict with -i insert\n  -k <key type>             default | hash | tree\n                            shortcut h: hash  t: tree    d: default\n  -S                        sequential, defaut is random\n  -V                        verify driver basic CRUD operations\n  -Q                        quiet mode, only show last result\n  -H                        show header, if -Q is set, then don't show header\n\nExample:\n  default: do ramdisk random mode test, insert 1M rows, query 1M times, update 1M times, delete 1M rows\n     -s m: do inmem random mode test,   insert 1M rows, query 1M times, update 1M times, delete 1M rows\n     -s d: do ondisk random mode test,  insert 1M rows, query 1M times, update 1M times, delete 1M rows\n     -i 500k -q 5m:            do ramdisk random mode test, insert 500K rows, query 5M times, updat 1M times, delete 500k rows\n     -i 10k -q 0 -u 0 -d 0:    do ramdisk random mode test, insert 10K rows\n     -r 10k -q 100k -u 0 -d 0: do ramdisk random mode test, query  100K times\n     -r 10k -q 0 -u 100k -d 0: do ramdisk random mode test, update 100K times\n     -r 10k -q 0 -u 0 -d 10k:  do ramdisk random mode test, delete 10K rows\n     -V:  verify db driver is ok\n

Note

ondisk DB is stored in current folder _benchmarkdb, ramdisk DB is stored in /tmp/_benchmarkdb. If you don't delete all rows in your test, you'd better remember to delete manually or do a simple default test or ./build.sh clean.

"},{"location":"docs/reference/crossbench/#build","title":"Build","text":"

Install sqlite3 and lmdb

sudo apt install libsqlite3-dev\nsudo apt install liblmdb-dev\n

Install CrossDB Library

sudo cp libxxxdb.so /usr/lib/\n

Build 

./build.sh\n

"},{"location":"docs/reference/crossbench/#output-example","title":"Output Example","text":"
./crossdb-bench.bin\nCrossDB Database Benchmark\n\nLocation: /tmp/_benchmarkdb/CrossDB\nType: RAMDISK\nKey: Default\nAccess: Random\n\nInsert rows: 1000000    Use time: 272466us    TPS: 3670182\n\nQuery Test: 1000000\n  Round: 1      Use Time: 231543us      QPS: 4318852\n  Round: 2      Use Time: 244874us      QPS: 4083732\n  Round: 3      Use Time: 240435us      QPS: 4159128\n  Round: 4      Use Time: 223147us      QPS: 4481350\n  Round: 5      Use Time: 221954us      QPS: 4505438\n  Round: 6      Use Time: 243468us      QPS: 4107315\n  Round: 7      Use Time: 240375us      QPS: 4160166\n  Round: 8      Use Time: 240878us      QPS: 4151479\n  Round: 9      Use Time: 229140us      QPS: 4364144\n  Round: 10     Use Time: 237540us      QPS: 4209817\nUpdate Test: 1000000\n  Round: 1      Use Time: 266616us      TPS: 3750712\n  Round: 2      Use Time: 281096us      TPS: 3557503\n  Round: 3      Use Time: 274047us      TPS: 3649009\n  Round: 4      Use Time: 265904us      TPS: 3760755\n  Round: 5      Use Time: 260426us      TPS: 3839862\n  Round: 6      Use Time: 265945us      TPS: 3760175\n  Round: 7      Use Time: 254995us      TPS: 3921645\n  Round: 8      Use Time: 274223us      TPS: 3646667\n  Round: 9      Use Time: 274394us      TPS: 3644394\n  Round: 10     Use Time: 290548us      TPS: 3441772\n\nDelete: 1000000 Use Time: 401207us      TPS: 2492478\n\nCrossDB Benchmark Result: RAMDISK Access=Random Key=Default Rows=1000000\n              DB            Rows          Insert           Query          Update          Delete\n         CrossDB         1000000         3670182         4450310         3840754         2492478\n
"},{"location":"docs/reference/crossbench/#benchmark-scripts","title":"Benchmark Scripts","text":"

You can use following scripts to do benchmark test. The parameter is the cpu core to test which is optional, but to make test reliable, you'd better do test or low load system.

./crossdb-benchmark.sh 7\n./sqlite-benchmark.sh 7\n./lmdb-benchmark.sh 7\n./stlmap-benchmark.sh 7\n./stlhmap-benchmark.sh 7\n

Output example 

On-Disk Benchmark Test\n              DB            Rows          Insert           Query          Update          Delete\n         CrossDB            1000          351741        22158982         2441267         2785515\n         CrossDB            1000          651465        22109811         2453643         2865329\n         CrossDB            1000          643500        22131423         2436465         2770083\n         CrossDB           10000         1523229        19490360         2385504         3036744\n         CrossDB           10000         1671402        19781879         2402664         3087372\n         CrossDB           10000         1538224        19753758         2383397         3049710\n         CrossDB          100000         1923409        14145718         2157846         2595178\n         CrossDB          100000         1833483        14079026         2152338         2472615\n         CrossDB          100000         1895123        13783290         2145765         1340213\n         CrossDB         1000000         1054961         6716753         1471987         1721339\n         CrossDB         1000000         1058803         6712856         1464983         1725965\n         CrossDB         1000000         1051711         6729894         1469273         1749135\n         CrossDB        10000000          598117         4020796         1202939         1172451\n         CrossDB        10000000          595118         3992847         1196629         1133848\n         CrossDB        10000000          596664         4025642         1189165         1119912\n         CrossDB        100000000         429711         3052691          924709          514820\n         CrossDB        100000000         394050         2984995         1081168          552174\n         CrossDB        100000000         372579         3002053         1084742          630738\nOn-RamDisk Benchmark Test\n              DB            Rows          Insert           Query          Update          Delete\n         CrossDB            1000         3134796        22019752        15952443         5076142\n         CrossDB            1000         1577287        22070616        16006938         5208333\n         CrossDB            1000         1623376        22059753        15972537         5128205\n         CrossDB           10000         3095975        19691196        14472099         6165228\n         CrossDB           10000         3454231        19662306        14318556         6265664\n         CrossDB           10000         2882675        19533775        13952115         6042296\n         CrossDB          100000         3547357        14193827        10452799         4989770\n         CrossDB          100000         3598934        13253040        10448443         5093205\n         CrossDB          100000         3744196        14170861        10174198         5261496\n         CrossDB         1000000         3740750         6740538         5310981         3029302\n         CrossDB         1000000         3708868         6753996         5488757         3017146\n         CrossDB         1000000         3682956         6748722         5491734         2986920\n         CrossDB        10000000         1936138         4006575         3570695         1803817\n         CrossDB        10000000         2331095         4023692         3568680         1858624\n         CrossDB        10000000         2271060         4018655         3566541         2015747\n         CrossDB        100000000        1466629         3007293         2734737          802871\n         CrossDB        100000000        1467744         2967245         2714797          815151\n         CrossDB        100000000        1442414         3004026         2734427          747224\nIn-Memory Benchmark Test\n              DB            Rows          Insert           Query          Update          Delete\n         CrossDB            1000         4016064        21937928        15225355         7812500\n         CrossDB            1000         1838235        22083004        15972435         7575757\n         CrossDB            1000         1461988        22109461        16017004         8333333\n         CrossDB           10000         5356186        19717491        14388566         7163323\n         CrossDB           10000         5254860        19530617        14567100         7385524\n         CrossDB           10000         3285151        19605321        14355210         7047216\n         CrossDB          100000         4159733        13979051        10180695         5471656\n         CrossDB          100000         3981367        14143811        10259367         5474052\n         CrossDB          100000         4039098        13841855        10042771         5144562\n         CrossDB         1000000         3907868         6742926         5475764         3044659\n         CrossDB         1000000         3972399         6741476         5378830         3184835\n         CrossDB         1000000         3963362         6761225         5484965         3167213\n         CrossDB        10000000         2489824         3997149         3556671         2038462\n         CrossDB        10000000         2480423         4013349         3555828         2070516\n         CrossDB        10000000         2486211         3913256         3449931         1956951\n         CrossDB        100000000        1696429         2807183         2546983         1468625\n         CrossDB        100000000        1789488         2903683         2613381         1480978\n         CrossDB        100000000        1783067         2886531         2627970         1492552\n

"},{"location":"docs/reference/crossbench/#new-driver","title":"New Driver","text":"

To make a new xxxdb driver, refer the driver template below.

#include \"benchmark.h\"\nconst char* xxxdb_create (char *dbname, dbtype_e type, keytype_e key, uint32_t flags) {\n// credata database or table, Primary Key: prefix+mask\n}\nvoid xxxdb_close ()\n{\n// close database\n}\nvoid xxxdb_begin ()\n{\n// begin transaction\n}\nvoid xxxdb_commit ()\n{\n// commit transaction\n}\nvoid xxxdb_insert (route_t *pRoute)\n{\n// insert route\n}\nbool xxxdb_query (route_t *pRoute)\n{\n// query route by Primary Key\n}\nvoid xxxdb_update (route_t *pRoute)\n{\n// update route set birth by Primary Key\n}\nvoid xxxdb_delete (route_t *pRoute)\n{\n// delete route by Primary Key\n}\n#define db_name     \"xxxDB\"\n#define db_create   xxxdb_create\n#define db_close    xxxdb_close\n#define db_insert   xxxdb_insert\n#define db_query    xxxdb_query\n#define db_update   xxxdb_update\n#define db_delete   xxxdb_delete\n#define db_begin    xxxdb_begin\n#define db_commit   xxxdb_commit\n#include \"benchmark.c\"\n

Verify the driver can work 

./xxxdb-bench.bin -V\n------ Verify Driver ------\nInsert route\nQuery route\n10.1.1.0/24->10.1.1.254 intf: eth1 metric: 1 birth: 1687848698 flags: 0x0\nUpdate route\nQuery route\n10.1.1.0/24->10.1.1.254 intf: eth1 metric: 1 birth: 1687848708 flags: 0x0\nDelete route\nQuery route\ncrossdb error: status = -4, no row found\n10.1.1.0/24\n------ Verify Done ------\n

"},{"location":"docs/reference/crossdb-cli/","title":"CrossDB CLI","text":"

The crossdb-cli CrossDB Command Line Interface (CLI) is the simplest way for users to manipulate and interact with CrossDB.

"},{"location":"docs/reference/crossdb-cli/#installation","title":"Installation","text":"

There is no need for additional installation steps to install CrossDB CLI as it is already included in package automatically.

"},{"location":"docs/reference/crossdb-cli/#execution","title":"Execution","text":"

To access the CrossDB CLI, you can execute crossdb-cli command-line utility from a terminal or double click in windows. 

./crossdb-cli\n   _____                   _____  ____      _\n  / ____|                 |  __ \\|  _ \\   _| |_\n | |     _ __ ___  ___ ___| |  | | |_) | |_   _|\n | |    | '__/ _ \\/ __/ __| |  | |  _ <    |_|\n | |____| | | (_) \\__ \\__ \\ |__| | |_) |\n  \\_____|_|  \\___/|___/___/_____/|____/ crossdb.org\n\nCrossDB 0.5.0  DB version 0.1  Little Endian  64bit\n\n============ Welcome to CrossDB Shell ============\n<help>: Help Info       <F1>: Shortcuts\n<exit>: Exit shell      <TAB>: Auto completion\n\nCrossDB>\n
After entering the CrossDB CLI, you can execute various SQL commands.

"},{"location":"docs/reference/crossdb-cli/#crossdb-cli-tips","title":"CrossDB CLI tips","text":"
  • Press TAB key to get auto completion
  • Use up and down keys to iterate the history of commands entered
  • Use F1 to get all shortcuts
  • ./crossdb-cli /tmp/_benchmarkdb/CrossDB open DB directly
  • Enter open /tmp/_benchmarkdb/CrossDB to oepn DB interactly
  • Enter show d to show all opened databases
  • Enter show t to show all tables in current database
  • Enter use <db> to switch currenbt database
  • Enter exit to exit CrossDB CLI
  • Append >> at end to redirect output to file
"},{"location":"docs/reference/crossdb-cli/#command-list","title":"Command List","text":"

Press TAB key to show it 

CrossDB>\nselect      Select records from TABLE\ninsert      Insert a record to TABLE\nreplace     Replace a record to TABLE (update if exists else insert)\nupdate      Update records in TABLE\ndelete      Delete records from TABLE\nbegin       Begin Transaction\ncommit      Commit Transaction\nrollback    Rollback Transaction\ncreate      Create INDEX, TABLE, DATABASE\ndrop        Drop INDEX, TABLE, DATABASE [!!! Caution !!!]\nshow        Show tables, databases, indexes, etc\nuse         Switch DATABASE\nopen        Open DATABASE\nclose       Close DATABASE\nhistory     Show history\nexit        Exit shell\n

"},{"location":"docs/reference/crossdb-cli/#database-command","title":"Database Command","text":""},{"location":"docs/reference/crossdb-cli/#open-databases","title":"Open Databases","text":"
./crossdb-cli /tmp/_benchmarkdb/CrossDB\n\nCrossDB> open /tmp/_benchmarkdb/CrossDB\n
"},{"location":"docs/reference/crossdb-cli/#switch-databases","title":"Switch Databases","text":"
CrossDB> use CrossDB\nCurrent Datatbase is changed to CrossDB\n
"},{"location":"docs/reference/crossdb-cli/#show-command","title":"Show Command","text":""},{"location":"docs/reference/crossdb-cli/#show-databases","title":"Show Databases","text":"
CrossDB> show d\nID  Name     Tables  Type     Share  Size  Path\n==  =======  ======  =======  =====  ====  =========================\n1   CrossDB  1       RamDisk  F      5792  /tmp/_benchmarkdb/CrossDB\nCurrent Database is CrossDB (\"use <dbname>\" to switch db)\n
"},{"location":"docs/reference/crossdb-cli/#show-tables","title":"Show Tables","text":"
CrossDB> show t\nID  Name   RowCount  RowLimit  RowSize  Columns  Indexes  TblSize\n==  =====  ========  ========  =======  =======  =======  =======\n1   route  0         0         48       7        1        5672\n
"},{"location":"docs/reference/crossdb-cli/#show-tables-indexes","title":"Show Tables Indexes","text":"
CrossDB> show indexes\nID  Table  Index    Columns      Primary  Unique  Type  RowNum  NodeNum  SlotUsed  Size\n==  =====  =======  ===========  =======  ======  ====  ======  =======  ========  ====\n1   route  PRIMARY  prefix,mask  T        T       HASH  0       0        0         1216\n
"},{"location":"docs/reference/crossdb-cli/#show-tables-status","title":"Show Tables Status","text":"
CrossDB> show status\nID  Name   RowCount  PoolCount  Cursor  TblSize  DatSize  IdxSize\n==  =====  ========  =========  ======  =======  =======  =======\n1   route  0         8          0       5672     784      1216\n
"},{"location":"docs/reference/crossdb-cli/#show-tables-counters","title":"Show Tables Counters","text":"
CrossDB> show counters\nID  Name   Scan  Insert   Update    Delete   InsertFail  UpdateFail  DeleteFail\n==  =====  ====  =======  ========  =======  ==========  ==========  ==========\n1   route  0     1000000  10000000  1000000  0           0           0\n
"},{"location":"docs/reference/crossdb-cli/#show-table-schema","title":"Show table schema","text":"
CrossDB> show create table route\ncreate table `route` (\n`prefix`       UINT(4,0)      format IPv4,\n`mask`         UINT(1,4),\n`nexthop`      UINT(4,8)      format IPv4,\n`metric`       UINT(1,12),\n`intf`         CHAR(16,13),\n`birth`        UINT(8,32)     format TIMESTAMP,\n`flags`        UINT(4,40)     format HEX,\nPRIMARY KEY (prefix,mask)\n) rec_size=48;\n
"},{"location":"docs/reference/crossdb-cli/#select-command","title":"Select Command","text":"
select < * | column1,columnm2,...> from <table> [ where columnA=valueA and columnB=valueB and ... ]\n[ order by columnX,columnY,... [desc|asc] ]\n[ limit <n> ] [ offset <m> ] | [limit offset,row_count]\n

Where operators: =, !=, >, >=, <, <=

Note

Default select * will only show none zero fields, select ** can show all fields. One field can only apper twice. Only AND is supported

CrossDB> select * from route limit 10\nID  prefix     mask  nexthop     metric  intf  birth                flags\n==  =========  ====  ==========  ======  ====  ===================  =====\n1   10.1.1.2   24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0\n2   10.1.1.55  24    10.1.1.254  1       eth1  2023-10-20T11:01:06  0x0\n3   10.1.1.18  24    10.1.1.254  1       eth1  2023-10-20T11:01:01  0x0\n4   10.1.1.90  24    10.1.1.254  1       eth1  2023-10-20T10:59:08  0x0\n5   10.1.1.37  24    10.1.1.254  1       eth1  2023-10-20T11:01:12  0x0\n6   10.1.1.27  24    10.1.1.254  1       eth1  2023-10-20T11:01:33  0x0\n7   10.1.1.61  24    10.1.1.254  1       eth1  2023-10-20T11:00:52  0x0\n8   10.1.1.95  24    10.1.1.254  1       eth1  2023-10-20T11:00:29  0x0\n9   10.1.1.46  24    10.1.1.254  1       eth1  2023-10-20T10:57:50  0x0\n10  10.1.1.75  24    10.1.1.254  1       eth1  2023-10-20T11:02:18  0x0\nQualified Rows: 100     Select Rows: 10    Use time 34us QPS 294117\n
CrossDB> select * from route where prefix=10.1.1.2\nID  prefix    mask  nexthop     metric  intf  birth                flags\n==  ========  ====  ==========  ======  ====  ===================  =====\n1   10.1.1.2  24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0\nQualified Rows: 1       Select Rows: 1    Use time 8us QPS 125000\n
CrossDB> select * from route where prefix>=10.1.1.10 and prefix<=10.1.1.20\nID  prefix     mask  nexthop     metric  intf  birth                flags\n==  =========  ====  ==========  ======  ====  ===================  =====\n1   10.1.1.18  24    10.1.1.254  1       eth1  2023-10-20T11:01:01  0x0\n2   10.1.1.20  24    10.1.1.254  1       eth1  2023-10-20T10:59:22  0x0\n3   10.1.1.14  24    10.1.1.254  1       eth1  2023-10-20T11:02:04  0x0\n4   10.1.1.11  24    10.1.1.254  1       eth1  2023-10-20T11:01:50  0x0\n5   10.1.1.12  24    10.1.1.254  1       eth1  2023-10-20T11:01:44  0x0\n6   10.1.1.19  24    10.1.1.254  1       eth1  2023-10-20T11:01:09  0x0\n7   10.1.1.16  24    10.1.1.254  1       eth1  2023-10-20T11:00:56  0x0\n8   10.1.1.17  24    10.1.1.254  1       eth1  2023-10-20T11:02:32  0x0\n9   10.1.1.15  24    10.1.1.254  1       eth1  2023-10-20T10:58:50  0x0\n10  10.1.1.10  24    10.1.1.254  1       eth1  2023-10-20T10:59:38  0x0\n11  10.1.1.13  24    10.1.1.254  1       eth1  2023-10-20T11:02:29  0x0\n
CrossDB> select * from route order by prefix\nID   prefix     mask  nexthop     metric  intf  birth                flags\n===  =========  ====  ==========  ======  ====  ===================  =====\n1    10.1.1.0   24    10.1.1.254  1       eth1  2023-10-20T11:00:23  0x0\n2    10.1.1.1   24    10.1.1.254  1       eth1  2023-10-20T11:02:36  0x0\n3    10.1.1.2   24    10.1.1.254  1       eth1  2023-10-20T11:02:05  0x0\n4    10.1.1.3   24    10.1.1.254  1       eth1  2023-10-20T11:02:21  0x0\n5    10.1.1.4   24    10.1.1.254  1       eth1  2023-10-20T11:02:52  0x0\n6    10.1.1.5   24    10.1.1.254  1       eth1  2023-10-20T11:01:36  0x0\n7    10.1.1.6   24    10.1.1.254  1       eth1  2023-10-20T11:02:25  0x0\n8    10.1.1.7   24    10.1.1.254  1       eth1  2023-10-20T11:00:43  0x0\n9    10.1.1.8   24    10.1.1.254  1       eth1  2023-10-20T11:01:20  0x0\n
"},{"location":"docs/reference/crossdb-cli/#dml-commands","title":"DML Commands","text":"

Warning

Database is accessed exclusively by single process with best performance. So if process is running and accessing the DB, please don't do the DML commands, otherwise the table may be corrupted. You can update fields not used in index, but be careful to do it.

"},{"location":"docs/reference/crossdb-cli/#insert-row","title":"Insert Row","text":"
insert into route set prefix=1.1.1.1 mask=24 nexthop=1.1.1.254 metric=5 intf='eth1'\n

Note

Where clause follows select syntax.

"},{"location":"docs/reference/crossdb-cli/#update-row","title":"Update Row","text":"
update route set metric=5 where prefix=1.1.1.1\n

Note

Where clause follows select syntax.

"},{"location":"docs/reference/crossdb-cli/#delete-row","title":"Delete Row","text":"
delete from route where prefix=1.1.1.1\n

Note

Where clause follows select syntax.

"},{"location":"products/CHANGELOG/","title":"Change Log","text":""},{"location":"products/CHANGELOG/#0.5.0","title":"0.5.0 (2023-06-26)","text":"

Features

  • CrossDB command line tool crossdb-cli is released
  • Optimize insert/update/query/delete performance
  • Add new API cross_matchCreate and cross_matchFree
  • DML APIs supports cross_fields_h and cross_match_h

Bug Fixes

"},{"location":"products/CHANGELOG/#0.4.0","title":"0.4.0 (2023-06-20)","text":"

Features

  • Support FreeBSD(X64)
  • Optimize insert/update/query/delete performance
  • Add new API cross_fieldsCreate and cross_fieldsFree

Bug Fixes

"},{"location":"products/CHANGELOG/#0.3.0","title":"0.3.0 (2023-06-13)","text":"

Features

  • Support MacOS (X64 and ARM64)
  • Change CROSS_DB_XXX to CROSS_XXX

Bug Fixes

  • cross_dbTblCreate flags CROSS_DB_RBTREE doesn't create Primary Key Index type correctly
"},{"location":"products/CHANGELOG/#0.2.0","title":"0.2.0 (2023-06-07)","text":"

Features

  • Support Windows
  • Support Linux ARM64

Bug Fixes

"},{"location":"products/CHANGELOG/#0.1.0","title":"0.1.0 (2023-06-03)","text":"
  • Initial release

Features

Bug Fixes

"},{"location":"products/download/","title":"Download","text":"

We provide precompiled binaries for CrossDB components. Download the latest release of CrossDB for your platform.

"},{"location":"products/download/#crossdb-v050","title":"CrossDB v0.5.0","text":"
  • 2023-06-26 Release notes
File Name OS Arch crossdb-0.5.0-linux-x64.tar.gz Linux X64 crossdb-0.5.0-linux-arm64.tar.gz Linux ARM64 crossdb-0.5.0-windows-x64.zip Windows X64 crossdb-0.5.0-windows-x86.zip Windows X86 crossdb-0.5.0-macos.tar.gz MacOS X64 and ARM64 crossdb-0.5.0-freebsd-x64.tar.gz FreeBSD X64"},{"location":"products/download/#crossdb-v040","title":"CrossDB v0.4.0","text":"
  • 2023-06-20 Release notes
File Name OS Arch crossdb-0.4.0-linux-x64.tar.gz Linux X64 crossdb-0.4.0-linux-arm64.tar.gz Linux ARM64 crossdb-0.4.0-windows-x64.zip Windows X64 crossdb-0.4.0-macos.tar.gz MacOS X64 and ARM64 crossdb-0.4.0-freebsd-x64.tar.gz FreeBSD X64"},{"location":"products/download/#crossdb-v030","title":"CrossDB v0.3.0","text":"
  • 2023-06-13 Release notes
File Name OS Arch crossdb-0.3.0-linux-x64.tar.gz Linux X64 crossdb-0.3.0-linux-arm64.tar.gz Linux ARM64 crossdb-0.3.0-windows-x64.zip Windows X64 crossdb-0.3.0-macos.tar.gz MacOS X64 and ARM64"},{"location":"products/download/#crossdb-v020","title":"CrossDB v0.2.0","text":"
  • 2023-06-07 Release notes
File Name OS Arch crossdb-0.2.0-linux-x64.tar.gz Linux X64 crossdb-0.2.0-linux-arm64.tar.gz Linux ARM64 crossdb-0.2.0-windows-x64.zip Windows X64"},{"location":"products/download/#crossdb-v010","title":"CrossDB v0.1.0","text":"
  • 2023-06-03 Release notes
File Name OS Arch crossdb-0.1.0-linux-x64.tar.gz Linux X64"},{"location":"products/price/","title":"Price and Buy","text":"Item Free Bronze Silver Gold Platinum Diamond DB/Process 2 16 32 64 128 128 Table/DB 16 128 256 512 1024 1024 Max Records/Table 200K 2M 10M 50M 200M 1B Max Row Size 1K 8K 16K 32K 64K 64K Max Columns/Table 256 1024 2048 3072 4096 4096 Max Index/Table 4 8 12 12 16 16 Max Trigger/Table 4 8 12 12 16 16 Max Foreign key/Table 4 8 12 12 16 16 CPU Core 1 2 4 8 16 512 Price $0 $100 $250 $500 $1000 $2000 Buy Free Buy now! Buy now! Buy now! Buy now! Buy now! Buy Free BuyNow BuyNow BuyNow BuyNow BuyNow"},{"location":"products/products/","title":"Products","text":""},{"location":"products/products/#crossdb-embedded-database-core","title":"CrossDB Embedded Database Core","text":"

CrossDB is a powerful High Performance Embedded Database which follows traditional RDBMS Model. It's developed to improve development efficiency for embedded programmers.

CrossDB combines On-Disk In-Memory and On-RamDisk data storage in a single embedded database system, so developers can optimize applications for speed and persistence, giving you flexibility to prioritize between performance, cost, power, and space-conserving storage options.

Get Started \ud83e\udded Read More \ud83d\udcdc

"}]} \ No newline at end of file +{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":"

CrossDBSuper High-performance Embedded and Server RDBMS\u2728

Quick Learn\ud83e\udded Benchmark \ud83d\udcdc

  • \ud83c\udf0c Simple

    CrossDB is easy to install and deploy. It has zero external dependencies and runs in-process in its host application or as a single binary with tiny footprint.

  • \u267b\ufe0f Portable

    CrossDB runs on Linux, macOS, Windows, BSD and all popular hardware architectures. It has idiomatic client APIs for major programming languages.

    Read more

  • \ud83d\ude80 Super Fast

    The hand-writing SQL parser and memory-oriented design architecture make the database run SQL at blazing speed.

  • \u26fd Hybrid Storage Mode

    CrossDB supports On-Disk database and In-Memory database(IMDB). It can also support Hybrid mode with some tables On-Disk and some In-Memory.

  • \ud83d\udd31 MySQL-like SQL

    Most SQL commands are similar with MySQL.

    Read more

  • \ud83d\udcae Server Mode

    Besides the Embedded RDBMS mode, CrossDB can also run in Embedded Server mode or run CrossDB as Standalone RDBMS server.

CSQL 
xdb_res_t   *pRes;\nxdb_row_t   *pRow;\nxdb_conn_t  *pConn = xdb_open (\":memory:\");\npRes = xdb_exec (pConn, \"CREATE TABLE student (id INT, name CHAR(16), age INT, class CHAR(16), score INT)\");\npRes = xdb_exec (pConn, \"INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)\");\npRes = xdb_exec (pConn, \"SELECT * from student\");\nwhile (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->row_meta, pRow);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\npRes = xdb_exec (pConn, \"UPDATE student set age=9 WHERE id = 2\");\npRes = xdb_exec (pConn, \"DELETE FROM student WHERE id = 3\");\nxdb_close (pConn);\n
CREATE TABLE student (id INT, name CHAR(16), age INT, class CHAR(16), score INT);\nINSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95);\nSELECT * FROM student;\nSELECT * FROM student WHERE id = 1;\nUPDATE student set age=9 WHERE id = 2;\nDELETE FROM student WHERE id = 3;\n

Note

This project was redesigned and rewritten from scratch. It's still in early development stage, so please DO NOT use in your project now.

"},{"location":"CHANGELOG/","title":"Change Log","text":""},{"location":"CHANGELOG/#060-2024-08-18","title":"0.6.0 (2024-08-18)","text":"

  • Initial refactor release

  • This project was redesigned and rewritten from scratch for over one year

  • Standard RDBMS model
  • New SQL APIs which can support more language
  • MySQL style SQL and shell, which will be easy to study.
"},{"location":"CHANGELOG/#050-2023-06-26","title":"0.5.0 (2023-06-26)","text":"

Features

  • CrossDB command line tool crossdb-cli is released
  • Optimize insert/update/query/delete performance
  • Add new API cross_matchCreate and cross_matchFree
  • DML APIs supports cross_fields_h and cross_match_h

Bug Fixes

"},{"location":"CHANGELOG/#040-2023-06-20","title":"0.4.0 (2023-06-20)","text":"

Features

  • Support FreeBSD(X64)
  • Optimize insert/update/query/delete performance
  • Add new API cross_fieldsCreate and cross_fieldsFree

Bug Fixes

"},{"location":"CHANGELOG/#030-2023-06-13","title":"0.3.0 (2023-06-13)","text":"

Features

  • Support MacOS (X64 and ARM64)
  • Change CROSS_DB_XXX to CROSS_XXX

Bug Fixes

  • cross_dbTblCreate flags CROSS_DB_RBTREE doesn't create Primary Key Index type correctly
"},{"location":"CHANGELOG/#020-2023-06-07","title":"0.2.0 (2023-06-07)","text":"

Features

  • Support Windows
  • Support Linux ARM64

Bug Fixes

"},{"location":"CHANGELOG/#010-2023-06-03","title":"0.1.0 (2023-06-03)","text":"
  • Initial release

Features

Bug Fixes

"},{"location":"community/","title":"Community","text":""},{"location":"community/#report-bugs","title":"Report Bugs","text":"

Github Issues

"},{"location":"community/#require-fetures","title":"Require Fetures","text":"

Github Fetures

"},{"location":"community/#discussion","title":"Discussion","text":"

Github Discussion

"},{"location":"community/#contribution","title":"Contribution","text":"

In order to keep CrossDB quality and high-performance, this project does not accept patches. If you would like to suggest a change and you include a patch as a proof-of-concept, that would be great. However, please do not be offended if we rewrite your patch from scratch.

Following contributions are welcome:

  • Language bindings: Python, Java, Go, CSharp, Javascript, PHP, etc
  • Test and report bugs
"},{"location":"faq/","title":"FAQ","text":""},{"location":"faq/#why-is-crossdb-so-fast","title":"Why is CrossDB so fast?","text":"
  • Use memory map to access DB data directly (so your DB must less than the main memory to get highest speed).
  • Use high-performance hash as the main index, and use the super fast wyhash as the hash function.
  • Hand-written SQL parser which is more than 10X faster than Flex/Bison parser tools.
  • High-performance read-write lock and multi-core optimization.
  • Execute multiple SQL statements in one API, which will imporve client-server DB performance dramatically.
  • High-efficient client-server protocols.
  • Avoid memory allocation/free as much as possible.
  • High-performance implementation code
"},{"location":"glossary/","title":"Glossary","text":""},{"location":"glossary/#acid","title":"ACID","text":"

ACID refers to the four key properties of a transaction: atomicity, consistency, isolation, and durability. Each of these properties is described below.

Atomicity means that either all the changes of an operation are performed, or none of them are.

Consistency means that transactions always bring the database from one consistent state to another.

Isolation means that a transaction in process is invisible to other transactions until it completes. This allows concurrent transactions to read and write data without sacrificing consistency.

Durability means that once a transaction is committed, it remains committed even in the event of a system failure.

"},{"location":"glossary/#crud","title":"CRUD","text":"

Acronym for \u201ccreate, read, update, delete\u201d, a common sequence of operations in database applications. Often denotes a class of applications with relatively simple database usage (basic DDL, DML and query statements in SQL) that can be implemented quickly in any language.

"},{"location":"glossary/#ddl","title":"DDL","text":"

Data definition language, a set of SQL statements for manipulating the database itself rather than individual table rows. Includes all forms of the CREATE, ALTER, and DROP statements. DDL statements automatically commit the current transaction; they cannot be rolled back.

"},{"location":"glossary/#dml","title":"DML","text":"

Data manipulation language, a set of SQL statements for performing SELECT, INSERT, UPDATE, and DELETE operations.

"},{"location":"glossary/#hash-index","title":"Hash Index","text":"

A type of index intended for queries that use equality operators, rather than range operators such as greater-than or BETWEEN.

"},{"location":"glossary/#imdb","title":"IMDB","text":"

IMDB(In-Memory database) is a type of database system that maintains data in memory, to avoid overhead due to disk I/O and translation between disk blocks and memory areas.

"},{"location":"glossary/#index","title":"Index","text":"

A data structure that provides a fast lookup capability for rows of a table, typically by forming a tree structure (B-tree) representing all the values of a particular column or set of columns.

"},{"location":"glossary/#information_schema","title":"INFORMATION_SCHEMA","text":"

The name of the database that provides a query interface to the CrpssDB data dictionary. (This name is defined by the ANSI SQL standard.) To examine information (metadata) about the database, you can query tables such as INFORMATION_SCHEMA.TABLES and INFORMATION_SCHEMA.COLUMNS, rather than using SHOW commands that produce unstructured output.

"},{"location":"glossary/#mvcc","title":"MVCC","text":"

MVCC(Multi-Version Concurrency Control) is a concurrency control mechanism in many databases. It processes the memory read by transactions to achieve concurrent access to DB, thereby avoiding blocking caused by conflicts between concurrent reads and writes.

"},{"location":"glossary/#oltp","title":"OLTP","text":"

Acronym for \u201cOnline Transaction Processing\u201d. A database system, or a database application, that runs a workload with many transactions, with frequent writes as well as reads, typically affecting small amounts of data at a time.

"},{"location":"glossary/#sql","title":"SQL","text":"

The Structured Query Language that is standard for performing database operations. Often divided into the categories DDL, DML, and queries.

"},{"location":"intro/","title":"Introduction","text":"

CrossDB is a super high-performance embedded and server RDBMS. It's developed for high performance scenarios with main memory can hold whole DB.

"},{"location":"intro/#features","title":"Features","text":"
  • Support Multiple OS Platforms: Linux/Windows/MacOS/FreeBSD etc
  • Support Multiple CPU ARCH: X86/ARM/PPC/MIPS etc
  • Support OnDisk/In-memory/RamDisk/Hybrid Storage
  • Support Standard RDBMS model
  • Support MySQL like SQL
  • Support Multiple databases
  • Support Primary Key and multiple Secondary Indexes
  • Support HASH and RBTREE(TBD) Index
  • Support Multi-columns Index
  • Support Exact Match, Leftmost Match(TBD), Range Match(TBD)
  • Support Standard ACID Transaction (begin/commit/rollback)
  • Support WAL for OnDisk storage(TBD)
  • Support Multiple Threads and Multiple Processes Access
  • Support Table level read-write lock
  • Support Reader-Writer MVCC
  • Support Embedded CrossDB Shell
  • Support Multi-Statments APIs
  • Support Prepared Statments APIs
  • Super High Performance
  • Very Simple: Simple header and library file
  • Zero Config: no complex config, real out-of-the-box
"},{"location":"intro/#use-cases","title":"Use Cases","text":"
  • You can use CrossDB In-Memory DB to manage Process Runtime Data to replace STL or hand-wrting data structures.
  • You can use CrossDB RamDisk DB to support Process Restartability, In-Service Software Upgrade(ISSU) easily.
  • You can use CrossDB OnDisk DB to store data on Disk/Flash/SDD.
  • You can use CrossDB to work as a super fast cache DB.
"},{"location":"about/buy/","title":"How to Buy","text":""},{"location":"about/privacy/","title":"Privacy Notice","text":""},{"location":"about/privacy/#introduction","title":"Introduction","text":"

CrossDB Software has designed its website so that, in general, you can browse the CrossDB website without being asked for any personal information. Exceptions to this are when you specifically ask for information, request an evaluation copy of our software or download CrossDB trial software.

This policy covers how CrossDB Software treats personal information that CrossDB may collect and receive via the CrossDB website. Your use of the CrossDB website constitutes acceptance of and agreement with the CrossDB Software Privacy Policy. CrossDB reserves the right to change the CrossDB Software Privacy Policy at any time without notice.

"},{"location":"about/privacy/#information-collection-and-use","title":"Information collection and use","text":"

CrossDB collects personal information when you request for information, request an evaluation of our software or when you download trial software. This information includes information about you such as name, business name and address, telephone and fax numbers, email address, operating system, and hardware environment. Providing information is voluntary. You may provide this information by filling out an on-line form, by email or other communication to CrossDB. However, you may not receive the product information, software evaluation or downloads if you choose not to provide the requested information.

CrossDB Software's use of the information you provided will be to send you the information that you requested. CrossDB Software may also use the information to provide you with additional information about our products and services or market research. The information may be used to follow up with you regarding information supplied, software you downloaded or evaluated.

CrossDB does not sell information collected from users of CrossDB website to other organizations. CrossDB may share the information collected with its subsidiaries and its affiliates such as partners and distributors.

"},{"location":"about/privacy/#general-information","title":"General Information","text":"

CrossDB web servers record standard information about CrossDB Web accesses such as IP addresses, your server name, browser type, etc. when you visit our website. CrossDB uses this for information about usage at CrossDB Software's website. The information submitted to CrossDB Software via the Web may be processed, stored and used outside the country where it was submitted.

Links to third party websites are provided for your convenience. By using these links, you will no longer be on the CrossDB Software website and the CrossDB Software Privacy Policy will no longer apply.

"},{"location":"about/privacy/#contact-us","title":"Contact us","text":"

If you have any question about the information you provided to us or about the CrossDB Software Privacy Policy, please contact us at support@crossdb.org

"},{"location":"about/refund/","title":"Refund Policy","text":"

Thank you for using CrossDB Database! We don't offer refund, please try it first then subscribe the product by month or year. You can cancel the subscription anytime as you want.

"},{"location":"about/terms/","title":"Terms of Service","text":"

CrossDB provides this Website for your information and use, subject to the terms set forth below. By accessing this Website, you agree to accept the following terms.

"},{"location":"about/terms/#general-terms","title":"General terms","text":"

In no event will CrossDB be liable for any damages including, without limitation, indirect, incidental, consequential, special or exemplary damages, that arise out of or relate to the use of or inability to use the CrossDB Website, even if CrossDB has been advised of the possibility of such damages.

Any and all information provided on CrossDB\u2019s Website is provided \u201cas is,\u201d with no warranty as to accuracy or content.

CrossDB does not adopt or endorse the views of any third party Websites linked to or from the CrossDB Website.

CrossDB reserves the right to change, modify, add or remove any of these terms at any time.

"},{"location":"admin/data-backup/","title":"Data Backup","text":""},{"location":"admin/data-backup/#dump-with-dump-statement","title":"Dump with DUMP statement","text":"
  • Dump everything
SQLC 
DUMP DATABASE school INTO 'school.sql';\n
xdb_exec (pConn, \"DUMP DATABASE school INTO 'school.sql'\");\n
  • Dump schema only
SQLC 
DUMP DATABASE school NODATA INTO 'school.sql';\n
xdb_exec (pConn, \"DUMP DATABASE school NODATA INTO 'school.sql'\");\n
  • Dump data only
SQLC 
DUMP DATABASE school NODROP NOCREATE INTO 'school.sql';\n
xdb_exec (pConn, \"DUMP DATABASE school NODROP NOCREATE INTO 'school.sql'\");\n
"},{"location":"admin/data-backup/#dump-with-crossdb-tool","title":"Dump with crossdb tool","text":"
  • Dump everything
crossdb -e \"DUMP DATABASE\" school > school.sql\n
crossdb -e \"DUMP DATABASE school INTO 'school.sql'\" school\n
  • Dump schema only
crossdb -e \"DUMP DATABASE school NODATA INTO 'school.sql'\" school\n
  • Dump data only
crossdb -e \"DUMP DATABASE school NODROP NOCREATE INTO 'school.sql'\" school\n
"},{"location":"admin/data-restore/","title":"Data Restore","text":""},{"location":"admin/data-restore/#load-with-source-statement","title":"Load with SOURCE statement","text":"SQLC 
SOURCE 'school.sql'\n
xdb_exec (pConn, \"SOURCE 'school.sql'\");\n
"},{"location":"admin/data-restore/#load-with-crossdb-tool","title":"Load with CrossDB tool","text":"
crossdb school < school.sql\n
cat school.sql | crossdb school\n
crossdb -e \"SOURCE 'school.sql'\" school\n
"},{"location":"admin/shell/","title":"CrossDB Shell","text":""},{"location":"admin/shell/#crossdb-tool","title":"CrossDB tool","text":"

This tool can

  • Open and operate local database
  • Connect and operate remote database
  • Work as standalone database server
$ crossdb -h\ncrossdb: option requires an argument -- 'h'\nUsage: xdb-cli [OPTIONS] [[path/]db_name]\n  -h                        Show this help\n  -S                        Server: Start in server mode\n  -h <ip>                   IP address to bind to or connect to\n  -P <port>                 Port to listen or connect\n  -D <datadir>              Server: Data directory to store databases\n  -u <user>                 Client user\n  -p                        Client password\n  -e <sql>                  Execute SQL statements and quite\n
"},{"location":"admin/shell/#open-db","title":"Open DB","text":"
crossdb school\n
"},{"location":"admin/shell/#open-db-and-execute-command","title":"Open DB and execute command","text":"
crossdb -e 'SELECT * FROM student WHERE id=1; SELECT * FROM student WHERE age=10' school\n
"},{"location":"admin/shell/#embedded-shell","title":"Embedded shell","text":"

You program can enter interactive shell use the SHELL statement.

SQLC 
SHELL\n
xdb_exec (pConn, \"SHELL\");\n
"},{"location":"admin/shell/#auto-completion","title":"Auto completion","text":"

Use TAB to do auto completion for SQL statements, keywords, db name, table name, field name, etc.

XDB> <TAB>\nCREATE      Create database, table, index, etc\nDROP        Drop database, table, index, etc\nALTER       Alter database, table, index, etc\nSELECT      Select rows from table\nINSERT      Insert rows into table\nUPDATE      Update rows in table\nDELETE      Delete rows from table\nBEGIN       Begin transaction\nCOMMIT      Commit transaction\nROLLBACK    Rollback transaction\nUSE         Switch Databae\nSHOW        Show database, table, index, etc\nDESCRIBE    Show Table Schema\nEXPLAIN     Show SELECT statement index selection\nSET         Config parameters\nOPEN        Open database\nCLOSE       Close database\nDUMP        Dump database\nSOURCE      Load SQL file\nSHELL       Enter interactive shell\nHELP        Help\n
XDB> S <TAB>\nSELECT    Select rows from table\nSHOW      Show database, table, index, etc\nSET       Config parameters\nSOURCE    Load SQL file\nSHELL     Enter interactive shell\nXDB> SE <TAB>\nSELECT    Select rows from table\nSET       Config parameters\nXDB> SEL <TAB>\nXDB> SELECT\n
XDB> SELECT <TAB>\nXDB> SELECT * <TAB>\nXDB> SELECT * FROM <TAB>\nstudent    teacher    book\nXDB> SELECT * FROM s<TAB>\nXDB> SELECT * FROM student\n
XDB> SELECT * FROM student <TAB>\nWHERE       ORDER BY    LIMIT       OFFSET      AND         id          name        age         class       score\ninfo\nXDB> SELECT * FROM student WHERE i <TAB>\nid      info\n
XDB> SHOW  <TAB>\nDATABASES    TABLES       INDEXES      COLUMNS      CREATE\n
"},{"location":"client/api-c/","title":"C APIs","text":""},{"location":"client/api-c/#apis-list","title":"APIs List","text":"API Descritpion xdb_conn_t* xdb_open (const char *path) Open a connection void xdb_close (xdb_conn_t *pConn) Close a connection xdb_res_t* xdb_exec (xdb_conn_t* pConn, const char *sql) Execute SQL statement xdb_res_t* xdb_pexec (xdb_conn_t* pConn, const char *sql, ...) Execute formatted SQL statement xdb_res_t* xdb_next_result (xdb_conn_t *pConn) Get next SQL statement result bool xdb_more_result (xdb_conn_t* pRes) Check is there more result void xdb_free_result (xdb_res_t* pRes) Free result set xdb_col_t* xdb_column_meta (uint64_t meta, uint16_t iCol) Fetch one row xdb_row_t* xdb_fetch_row (xdb_res_t* pRes) Fetch one row int xdb_column_int (uint64_t meta, void *pRow, uint16_t iCol) Get int column from row float xdb_column_float (uint64_t meta, void *pRow, uint16_t iCol) Get float/double column from row const char * xdb_column_str (uint64_t meta, void *pRow, uint16_t iCol) Get string column from row xdb_stmt_t* xdb_stmt_prepare (xdb_conn_t* pConn, const char *sql) Prepare statement xdb_ret xdb_bind_int (xdb_stmt_t *pStmt, uint16_t para_id, int val) Bind int value xdb_ret xdb_bind_float (xdb_stmt_t *pStmt, uint16_t para_id, float val) Bind float value xdb_ret xdb_bind_str (xdb_stmt_t *pStmt, uint16_t para_id, const char *str) Bind string value xdb_res_t* xdb_stmt_exec (xdb_stmt_t *pStmt) Execute prepared SQL statement void xdb_stmt_close (xdb_stmt_t *pStmt) Free prepared SQL statement xdb_ret xdb_begin (xdb_conn_t* pConn) Begin transaction xdb_ret xdb_commit (xdb_conn_t* pConn) Commit transaction xdb_ret xdb_rollback (xdb_conn_t* pConn) Rollback transaction int xdb_print_row (uint64_t meta, void *pRow, int format) Print row to console const char * xdb_errmsg (xdb_res_t *pRes) Get error/information message const char * xdb_version () Get CrossDB version string"},{"location":"client/api-c/#xdb_open","title":"xdb_open","text":"

Open a connection and create/open a Database.

xdb_conn_t*\nxdb_open (const char *path);\n\n// TBD\nxdb_conn_t*\nxdb_open2 (const char *path, uint32_t flags);\n
  • If path is :memory: will create or open a memory database, else create or open a on-disk database. This database will be the default DB of this connection.
  • If path is NULL: Only open a connection. User has to use OPEN DATABASE '[path/]db_name' or CREATE DATABASE '[path/]db_name' or USE DATABASE db_name.
  • One thread can use only one connection. One connection can only be used by one thread
  • All opened databases are shared with all opened connections, and they can USE DATABASE db_name to switch connection default DB.
"},{"location":"client/api-c/#xdb_close","title":"xdb_close","text":"

Create a connection.

void\nxdb_close (xdb_conn_t *pConn);\n
"},{"location":"client/api-c/#xdb_exec","title":"xdb_exec","text":"

Execute SQL statement and return result set.

xdb_res_t*\nxdb_exec (xdb_conn_t* pConn, const char *sql);\n\nxdb_res_t*\nxdb_exec2 (xdb_conn_t *pConn, const char *sql, int len);\n
  • A valid xdb_res_t pointer is returned always.
  • row count: pRes->row_count
  • column count: pRes->col_count
  • column meta: pRes->row_meta, use xdb_col_t* xdb_column_meta (uint64_t meta, uint16_t iCol) to get column meta.
"},{"location":"client/api-c/#xdb_pexec","title":"xdb_pexec","text":"

Execute formatted SQL statement and return result set.

xdb_res_t*\nxdb_pexec (xdb_conn_t *pConn, const char *sql, ...);\n
  • A valid xdb_res_t pointer is returned always.

Note

If sql contains %, should use escape \\%.

"},{"location":"client/api-c/#xdb_next_result","title":"xdb_next_result","text":"

Get next SQL statement result 

xdb_res_t*\nxdb_next_result (xdb_conn_t *pConn);\n

"},{"location":"client/api-c/#xdb_more_result","title":"xdb_more_result","text":"

Check is there more result 

bool\nxdb_more_result (xdb_conn_t *pConn);\n

"},{"location":"client/api-c/#xdb_free_result","title":"xdb_free_result","text":"

Free result set 

void\nxdb_free_result (xdb_res_t *pRes);\n

Note

You only need to call xdb_free_result for statements more than 0 rows.

"},{"location":"client/api-c/#xdb_column_meta","title":"xdb_column_meta","text":"

Get column meta.

const xdb_col_t* \nxdb_column_meta (uint64_t meta, uint16_t iCol)\n

Note

This is an inline function.

"},{"location":"client/api-c/#xdb_fetch_row","title":"xdb_fetch_row","text":"

Fetch one row and return row list array.

xdb_row_t*\nxdb_fetch_row (xdb_res_t *pRes);\n
"},{"location":"client/api-c/#xdb_column_int","title":"xdb_column_int","text":"

Get int column from row. 

int \nxdb_column_int (uint64_t meta, void *pRow, uint16_t iCol);\n\nint64_t \nxdb_column_int64 (uint64_t meta, void *pRow, uint16_t iCol);\n

Note

If you know the detailed format, you can access the pointer directly. *(int8_t*)pVal[iCol], *(int16_t*)pVal[iCol], *(int32_t*)pVal[iCol], *(int64_t*)pVal[iCol]

"},{"location":"client/api-c/#xdb_column_float","title":"xdb_column_float","text":"

Get float column from row.

float\nxdb_column_float (uint64_t meta, xdb_row_t *pRow, uint16_t iCol);\n\ndouble \nxdb_column_double (uint64_t meta, void *pRow, uint16_t iCol);\n

Note

If you know the detailed format, you can access the pointer directly. *(float*)pVal[iCol], *(double*)pVal[iCol]

"},{"location":"client/api-c/#xdb_column_str","title":"xdb_column_str","text":"

Get string column from row.

const char*\nxdb_column_str (uint64_t meta, xdb_row_t *pRow, uint16_t iCol);\n\nconst char*\nxdb_column_str2 (uint64_t meta, xdb_row_t *pRow, uint16_t iCol, int *pLen);\n

Note

You can access the pointer directly: *(const char*)pVal[iCol], get the length *(uint16_t*)(pVal[iCol]-2)

"},{"location":"client/api-c/#xdb_stmt_prepare","title":"xdb_stmt_prepare","text":"
xdb_stmt_t* \nxdb_stmt_prepare (xdb_conn_t* pConn, const char *sql);\n
"},{"location":"client/api-c/#xdb_bind_int","title":"xdb_bind_int","text":"

Binds an int value to the prepared statement at the specified parameter index.

xdb_ret\nxdb_bind_int (xdb_stmt_t *pStmt, uint16_t para_id, int val);\n\nxdb_ret\nxdb_bind_int64 (xdb_stmt_t *pStmt, uint16_t para_id, int64_t val);\n
"},{"location":"client/api-c/#xdb_bind_float","title":"xdb_bind_float","text":"

Binds a double value to the prepared statement at the specified parameter index.

xdb_ret\nxdb_bind_float (xdb_stmt_t *pStmt, uint16_t para_id, float val);\n\nxdb_ret\nxdb_bind_double (xdb_stmt_t *pStmt, uint16_t para_id, double val)\n
"},{"location":"client/api-c/#xdb_bind_str","title":"xdb_bind_str","text":"

Binds a string value to the prepared statement at the specified parameter index.

xdb_ret\nxdb_bind_str (xdb_stmt_t *pStmt, uint16_t id, const char *str, int len);\n\nxdb_ret\nxdb_bind_str2 (xdb_stmt_t *pStmt, uint16_t para_id, const char *str, int len)\n
"},{"location":"client/api-c/#xdb_stmt_exec","title":"xdb_stmt_exec","text":"

Execute a prepared statement.

xdb_res_t*\nxdb_stmt_exec (xdb_stmt_t *pStmt);\n
"},{"location":"client/api-c/#xdb_stmt_close","title":"xdb_stmt_close","text":"

Close a prepared statement.

void\nxdb_stmt_close (xdb_stmt_t *pStmt);\n
"},{"location":"client/api-c/#xdb_begin","title":"xdb_begin","text":"

Begin a transaction.

xdb_ret\nxdb_begin (xdb_conn_t* pConn);\n
  • For embedded local connection, always return XDB_OK.
"},{"location":"client/api-c/#xdb_commit","title":"xdb_commit","text":"

Commit a transaction.

xdb_ret\nxdb_commit (xdb_conn_t* pConn);\n
  • For embedded local connection, always return XDB_OK.
"},{"location":"client/api-c/#xdb_rollback","title":"xdb_rollback","text":"

Rollback a transaction.

xdb_ret\nxdb_rollback (xdb_conn_t* pConn);\n
  • For embedded local connection, always return XDB_OK.
"},{"location":"client/api-c/#xdb_print_row","title":"xdb_print_row","text":"

Print row to console.

int \nxdb_print_row (uint64_t meta, xdb_row_t *pRow, int format);\n
"},{"location":"client/api-c/#xdb_errmsg","title":"xdb_errmsg","text":"

Get error/information message in result.

const char *\nxdb_errmsg (xdb_res_t *pRes);\n
"},{"location":"client/api-c/#xdb_version","title":"xdb_version","text":"

Get CrossDB version string.

const char*\nxdb_version();\n
"},{"location":"client/api-c/#types","title":"Types","text":""},{"location":"client/api-c/#xdb_errno_e","title":"xdb_errno_e","text":"

Error Code 

typedef enum {\n    XDB_OK,\n    XDB_ERROR,\n    XDB_E_PARAM,\n    XDB_E_STMT,\n    XDB_E_NODB,\n    XDB_E_NOTFOUND,\n    XDB_E_EXISTS,\n    XDB_E_FULL,\n    XDB_E_CONSTRAINT,\n    XDB_E_AUTH,\n    XDB_E_MEMORY,\n    XDB_E_FILE,\n    XDB_E_SOCK,\n} xdb_errno_e;\n

"},{"location":"client/api-c/#xdb_type_t","title":"xdb_type_t","text":"

Data Types 

typedef enum {\n    XDB_TYPE_NULL       = 0,  // 1 bit\n    XDB_TYPE_TINYINT    = 1,  // 1 byte\n    XDB_TYPE_SMALLINT   = 2,  // 2 bytes\n    XDB_TYPE_INT        = 3,  // 4 bytes\n    XDB_TYPE_BIGINT     = 4,  // 8 bytes\n    XDB_TYPE_UTINYINT   = 5,  // 1 byte\n    XDB_TYPE_USMALLINT  = 6,  // 2 bytes\n    XDB_TYPE_UINT       = 7,  // 4 bytes\n    XDB_TYPE_UBIGINT    = 8,  // 8 bytes\n    XDB_TYPE_FLOAT      = 9,  // 4 bytes\n    XDB_TYPE_DOUBLE     = 10, // 8 bytes\n    XDB_TYPE_TIMESTAMP  = 11, // 8 bytes\n    XDB_TYPE_CHAR       = 12, // fixed-length string(at most 65535 byte)\n    XDB_TYPE_BINARY     = 13, // fixed-length binary(at most 65535 byte)\n    XDB_TYPE_VCHAR      = 14, // varied-length string(at most 65535 byte)\n    XDB_TYPE_VBINARY    = 15, // varied-length binary(at most 65535 byte)\n    XDB_TYPE_MAX        = 21\n} xdb_type_t;\n

"},{"location":"client/api-c/#structures","title":"Structures","text":""},{"location":"client/api-c/#xdb_res_t","title":"xdb_res_t","text":"

Result Set 

typedef struct xdb_res_t {\n    uint32_t    len_type;       // MSB 4bit are type xdb_restype_t\n    uint16_t    errcode;        // 4\n    uint16_t    status;         // 6 xdb_status_t\n\n    uint32_t    meta_len;       // 8\n    uint16_t    col_count;      // 12\n    uint8_t     stmt_type;      // 14 SQL type(create/delete/drop/show/select/delete/update...)\n    uint8_t     rsvd;\n\n    uint64_t    row_count;      // 2*8 SELECT/SHOW\n    uint64_t    affected_rows;  // 3*8 INSERT/UPDATE/DELETE\n    uint64_t    insert_id;      // 4*8 INSERT\n    uint64_t    col_meta;       // 5*8 xdb_meta_t, <ptr:ptr off: 0 following is meta>\n    uint64_t    row_data;       // 6*8 xdb_rowlist_t, ptr: base ptr or error str or infomation xdb_msg_t\n    uint64_t    data_len;       // 7*8\n} xdb_res_t;\n

"},{"location":"client/api-c/#xdb_msg_t","title":"xdb_msg_t","text":"

Return Message 

typedef struct {\n    uint32_t    len_type;       // LSB 4bit are type\n    uint16_t    len;\n    char        msg[];\n} xdb_msg_t;\n

"},{"location":"client/api-c/#xdb_meta_t","title":"xdb_meta_t","text":"

Query Meta information 

typedef struct {\n    uint32_t    len_type;       // LSB 4bit are type\n    uint16_t    col_count;      // 3*4\n    uint16_t    null_off;       // 3*4+2\n    uint16_t    row_size;\n    uint16_t    rsvd;\n    uint64_t    col_list;       // xdb_col_t list\n    xdb_col_t   cols[];\n} xdb_meta_t;\n

"},{"location":"client/api-c/#xdb_col_t","title":"xdb_col_t","text":"

Query Column information 

typedef struct {\n    uint16_t    col_len;    // colum total len\n    uint8_t     col_type;   // xdb_type_t\n    uint8_t     col_rsvd;\n    uint32_t    col_off;\n    uint16_t    col_rsvd2;\n    uint8_t     col_nmlen;\n    char        col_name[];\n} xdb_col_t;\n

"},{"location":"client/api-c/#xdb_rowdat_t","title":"xdb_rowdat_t","text":"

Query row data information 

typedef struct {\n    uint32_t    len_type;       // LSB 4bit are type\n    uint8_t     rowdat[];\n} xdb_rowdat_t;\n

"},{"location":"client/api-c/#xdb_rowlist_t","title":"xdb_rowlist_t","text":"
typedef uint64_t xdb_row_t; // xdb_rowdat_t\n\ntypedef struct {\n    uint32_t    rl_count;\n    uint32_t    rl_curid;\n    xdb_row_t   rl_pRows[];\n} xdb_rowlist_t;\n
"},{"location":"client/api-python/","title":"Python","text":"

TBD

"},{"location":"develop/connect/","title":"Connect to CrossDB","text":"
  • One thread can use only one connection. One connection can only be used by one thread.
  • All opened databases are shared with all opened connections, and they can USE DATABASE db_name to switch connection's default DB.
"},{"location":"develop/connect/#c-apis","title":"C APIs","text":"
  • Open a connection
xdb_conn_t *pConn = xdb_open (NULL);\n

Note

User has to use OPEN DATABASE '[path/]db_name' or CREATE DATABASE '[path/]db_name' or USE DATABASE db_name.

  • Open a connection and create a default memory database if not exist
xdb_conn_t *pConn = xdb_open (\":memory:\");\n
  • Open a connection and create a on-disk database school if not exist in current folder as the default database
xdb_conn_t *pConn = xdb_open (\"school\");\n
"},{"location":"develop/ddl/","title":"Data Model","text":""},{"location":"develop/ddl/#create-and-use-a-database","title":"Create and Use a Database","text":"SQLC 
-- Create database in current folder\nCREATE DATABASE school;\n-- Create database in specified folder (must exist)\nCREATE DATABASE '/var/dbpath/school';\n-- Create in-memory database \nCREATE DATABASE test ENGINE=memory;\n
// Create database in current folder\nxdb_res_t *pRes = xdb_exec (pConn, \"CREATE DATABASE school\");\n// Create database in specified folder (must exist)\nxdb_res_t *pRes = xdb_exec (pConn, \"CREATE DATABASE '/var/dbpath/school'\");\n// Create in-memory database \nxdb_res_t *pRes = xdb_exec (pConn, \"CREATE DATABASE test ENGINE=memory\");\n
"},{"location":"develop/ddl/#create-a-table","title":"Create a Table","text":"SQLC 
CREATE TABLE IF NOT EXISTS student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(256), INDEX (name));\nCREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(256), INDEX (name));\nCREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name));\n
xdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE TABLE IF NOT EXISTS student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(256), INDEX (name))\");\nxdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(256), INDEX (name))\");\nxdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name))\");\n
"},{"location":"develop/ddl/#create-index","title":"Create Index","text":"SQLC 
CREATE INDEX idx_name ON student (age);\nCREATE INDEX idx_name ON teacher (age);\nCREATE INDEX idx_name ON student (author);\n
xdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE INDEX idx_name ON student (age)\");\nxdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE INDEX idx_name ON teacher (age)\");\nxdb_res_t *pRes = pRes = xdb_exec (pConn, \"CREATE INDEX idx_name ON student (author)\");\n
"},{"location":"develop/dml/","title":"Data Manipulation","text":""},{"location":"develop/dml/#insert-rows","title":"Insert Rows","text":"SQLC 
INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95);\nINSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')\", 'He is a boy.\\nHe likes playing football.\\nWe all like him!');\n
pRes = xdb_exec (pConn, \"INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)\");\npRes = xdb_pexec (pConn, \"INSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')\", \"He is a boy.\\nHe likes playing football.\\nWe all like him!\");\n
"},{"location":"develop/dml/#update-rows","title":"Update Rows","text":"

TBD

"},{"location":"develop/dml/#delete-rows","title":"Delete Rows","text":"

TBD

"},{"location":"develop/multi-stmts/","title":"Multi-Statements","text":"

TBD

"},{"location":"develop/prep-stmts/","title":"Prepared Statements","text":"

TBD

"},{"location":"develop/query/","title":"Query Rows","text":"

TBD

"},{"location":"develop/transaction/","title":"Transaction","text":""},{"location":"develop/transaction/#reader-writeer-mvcc","title":"Reader-Writeer MVCC","text":"

Current CrossDB uses table-level readwrite lock, only one writer connection thread can modify the table, other writer connection threads will be blocked. However the reader connection threads won't be blocked and they'll see the old values. When the writer connection change is committed, the new value is visible by these reader connections (read committed). In short writer before commit will not block readers to read old values

"},{"location":"develop/transaction/#auto-commit","title":"Auto-commit","text":"

TBD

"},{"location":"develop/transaction/#begin","title":"Begin","text":"

TBD

"},{"location":"develop/transaction/#commit","title":"Commit","text":"

TBD

"},{"location":"develop/transaction/#rollback","title":"Rollback","text":"

TBD

"},{"location":"get-started/bench/","title":"Example and Bench","text":""},{"location":"get-started/bench/#example","title":"Example","text":"
crossdb$ make example\n=== Select all 7 rows\nid=1 name='jack' age=10 class='3-1' score=90.000000 info=''\nid=2 name='tom' age=11 class='2-5' score=91.000000 info=''\nid=3 name='jack' age=11 class='1-6' score=92.000000 info=''\nid=4 name='rose' age=10 class='4-2' score=90.000000 info=''\nid=5 name='tim' age=10 class='3-1' score=95.000000 info=''\nid=6 name='Tony' age=10 class='3-1' score=95.000000 info='He is a boy.\nHe likes playing football.\nWe all like him!'\nid=7 name='Wendy' age=10 class='3-1' score=95.000000 info='She is a girl.\nShe likes cooking.\nWe all love her!'\n\n=== Update age = 9 for id = 2\nselect 1 rows\nid=2 name='tom' age=9 class='2-5' score=91.000000\nid=2 name='tom' age=9 class='2-5' score=91.000000\nid=2 name='tom' age=9 class='2-5' score=91.000000\n\n=== Delete id = 3\nselect 0 rows\n=== select 1 rows\nCOUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667\nCOUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667\nCOUNT(*)=6 MIN(score)=90.000000 MAX(score)=95.000000 SUM(score)=556.000000 AVG(score)=92.666667\n\n=== Muti-Statements\nCOUNT(*)=6\nid=2 name='tom'\n   _____                   _____  ____      _\n  / ____|                 |  __ \\|  _ \\   _| |_\n | |     _ __ ___  ___ ___| |  | | |_) | |_   _|\n | |    | '__/ _ \\/ __/ __| |  | |  _ <    |_|\n | |____| | | (_) \\__ \\__ \\ |__| | |_) |  0.6.0\n  \\_____|_|  \\___/|___/___/_____/|____/ crossdb.org\n\n============ Welcome to CrossDB Shell ============\n<help>: Help Info       <F1>: Shortcuts\n<exit>: Exit shell      <TAB>: Auto completion\n\nXDB> exit\n
"},{"location":"get-started/bench/#bench-test","title":"Bench Test","text":"

Test with 1000 rows

crossdb/bench/c$ ./a.out 1000\n********************** INSERT 1000 with SQL **********************\nUse time 763ns, QPS 1310615\n********************** Random LKUP 5000000 with SQL **********************\nUse time 951987ns, QPS 5252172\nUse time 921033ns, QPS 5428687\nUse time 915605ns, QPS 5460870\nUse time 926288ns, QPS 5397889\n********************** UPDATE 1000 with SQL **********************\nUse time 760ns, QPS 1315789\n********************** DELETE 1000 with SQL **********************\nUse time 438ns, QPS 2283105\n\n********************** INSERT 1000 with SQL **********************\nUse time 687ns, QPS 1455604\n********************** Random LKUP 10000000 with Prepared STMT **********************\nUse time 294717ns, QPS 33930855\nUse time 297268ns, QPS 33639678\nUse time 298728ns, QPS 33475268\nUse time 295278ns, QPS 33866390\n********************** UPDATE 1000 with Prepared SMTT **********************\nUse time 576ns, QPS 1736111\n********************** DELETE 1000 with Prepared SQL **********************\nUse time 252ns, QPS 3968253\n

Test with 1000,1000 rows

crossdb$ make bench\nor\ncrossdb/bench/c$ ./a.out\n\n********************** INSERT 1000000 with SQL **********************\nUse time 539342ns, QPS 1854111\n********************** Random LKUP 1000000 with SQL **********************\nUse time 336044ns, QPS 2975800\nUse time 333927ns, QPS 2994666\nUse time 329699ns, QPS 3033069\nUse time 331632ns, QPS 3015390\n********************** UPDATE 1000000 with SQL **********************\nUse time 579945ns, QPS 1724301\n********************** DELETE 1000000 with SQL **********************\nUse time 405004ns, QPS 2469111\n\n********************** INSERT 1000000 with SQL **********************\nUse time 527486ns, QPS 1895784\n********************** Random LKUP 10000000 with Prepared STMT **********************\nUse time 1678936ns, QPS 5956153\nUse time 1668971ns, QPS 5991715\nUse time 1670993ns, QPS 5984465\nUse time 1652815ns, QPS 6050283\n********************** UPDATE 1000000 with Prepared SMTT **********************\nUse time 381320ns, QPS 2622469\n********************** DELETE 1000000 with Prepared SQL **********************\nUse time 232181ns, QPS 4306984\n
"},{"location":"get-started/install/","title":"Build and Install","text":""},{"location":"get-started/install/#build","title":"Build","text":"
crossdb$ make\nmake build                Build crossdb library and tool\nmake debug                Build crossdb library and tool with debug\nmake run                  Run crossdb tool\nmake clean                Clean build result\nmake install              Install crossdb(lib&tool&header) to Linux & FreeBSD\nmake uninstall            Uninstall crossdb from Linux & FreeBSD\nmake installmacos         Install crossdb(lib&tool&header) to MacOS\nmake uninstallmacos       Uninstall crossdb from MacOS\nmake example              Build and run example (need to install crossdb first)\nmake bench                Build and run bench test (need to install crossdb first)\n
"},{"location":"get-started/install/#linuxfreebsd","title":"Linux/FreeBSD","text":"
make build\nsudo make install\n
"},{"location":"get-started/install/#macos","title":"MacOS","text":"
make build\nsudo make installmacos\n
"},{"location":"get-started/install/#windows","title":"Windows","text":"

You need to install MINGW64 to build. Then set gcc path to system environment variables Path and make sure gcc can run.

build\n
"},{"location":"get-started/tutorial/","title":"Turorial","text":"
  • Open conection and create default memory database
xdb_res_t   *pRes;\nxdb_row_t   *pRow;\n\nxdb_conn_t  *pConn = xdb_open (\":memory:\");\n
  • Create Table
pRes = xdb_exec (pConn, \"CREATE TABLE student (id INT PRIMARY KEY, name CHAR(16), age INT, class CHAR(16), score FLOAT, info CHAR(255))\");\nXDB_CHECK(pRes, printf (\"Can't create table student\\n\"); goto error;);\npRes = xdb_exec (pConn, \"CREATE TABLE IF NOT EXISTS teacher (id INT PRIMARY KEY, name CHAR(16), age INT, info CHAR(255), INDEX (name))\");\nXDB_CHECK(pRes, printf (\"Can't create table teacher\\n\"); goto error;);\npRes = xdb_exec (pConn, \"CREATE TABLE IF NOT EXISTS book (id INT PRIMARY KEY, name CHAR(64), author CHAR(32), count INT, INDEX (name))\");\nXDB_CHECK(pRes, printf (\"Can't create table book\\n\"); goto error;);\n
  • Insert Rows
pRes = xdb_exec (pConn, \"INSERT INTO student (id,name,age,class,score) VALUES (1,'jack',10,'3-1',90),(2,'tom',11,'2-5',91),(3,'jack',11,'1-6',92),(4,'rose',10,'4-2',90),(5,'tim',10,'3-1',95)\");\nXDB_CHECK(pRes, printf (\"Can't insert table student\\n\"); goto error;);\npRes = xdb_pexec (pConn, \"INSERT INTO student (id,name,age,class,score,info) VALUES (6,'Tony',10,'3-1',95,'%s')\", \"He is a boy.\\nHe likes playing football.\\nWe all like him!\");\nXDB_CHECK(pRes, printf (\"Can't insert table student\\n\"); goto error;);\npRes = xdb_pexec (pConn, \"INSERT INTO student (id,name,age,class,score,info) VALUES (7,'Wendy',10,'3-1',95,'%s')\", \"She is a girl.\\nShe likes cooking.\\nWe all love her!\");\nXDB_CHECK(pRes, printf (\"Can't insert table student\\n\"); goto error;);\npRes = xdb_exec (pConn, \"INSERT INTO teacher (id,name,age) VALUES (1,'Tomas',40),(2,'Steven',50),(3,'Bill',31),(4,'Lucy',29)\");\nXDB_CHECK(pRes, printf (\"Can't insert table teacher\\n\"); goto error;);\npRes = xdb_exec (pConn, \"INSERT INTO book (id,name,author,count) VALUES (1,'Romeo and Juliet','Shakespeare',10),(2,'Pride and Prejudice','Austen',5),(3,'Great Expectations','Dickens',8),(4,'Sorrows of Young Werther','Von Goethe',4)\");\nXDB_CHECK(pRes, printf (\"Can't insert table book\\n\"); goto error;);\n
  • Select Rows
pRes = xdb_exec (pConn, \"SELECT * from student\");\nprintf (\"=== Select all %d rows\\n\", (int)pRes->row_count);\nwhile (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n    printf (\"id=%d name='%s' age=%d class='%s' score=%f\\n\", \n        xdb_column_int (pRes->col_meta, pRow, 0), \n        xdb_column_str (pRes->col_meta, pRow, 1), \n        xdb_column_int (pRes->col_meta, pRow, 2), \n        xdb_column_str (pRes->col_meta, pRow, 3), \n        xdb_column_float(pRes->col_meta, pRow, 4));\n    printf (\"id=%d name='%s' age=%d class='%s' score=%f\\n\", \n        *(int*)pRow[0], \n        (char*)pRow[1], \n        *(int*)pRow[2], \n        (char*)pRow[3], \n        *(float*)pRow[4]);\n}\nxdb_free_result (pRes);\n
  • Update Rows
printf (\"\\n=== Update age = 9 for id = 2\\n\");\npRes = xdb_exec (pConn, \"UPDATE student set age=9 WHERE id = 2\");\nXDB_CHECK(pRes, printf (\"Can't update id=%d\\n\",2); goto error;);\n\npRes = xdb_exec (pConn, \"SELECT id,name,age,class,score from student WHERE id = 2\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nwhile (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\n
  • Delete Rows
printf (\"\\n=== Delete id = 3\\n\");\npRes = xdb_exec (pConn, \"DELETE FROM student WHERE id = 3\");\nXDB_CHECK(pRes, printf (\"Can't delete id=%d\\n\",3); goto error;);\n\npRes = xdb_exec (pConn, \"SELECT * from student WHERE id = 3\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nwhile (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\n
  • Aggregation function
printf (\"\\n=== AGG COUNT,MIN,MAX,SUM,AVG\\n\");\npRes = xdb_exec (pConn, \"SELECT COUNT(*),MIN(score),MAX(score),SUM(score),AVG(score) FROM student\");\nprintf (\"=== select %d rows\\n\", (int)pRes->row_count);\nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n    printf (\"COUNT(*)=%d MIN(score)=%f MAX(score)=%f SUM(score)=%f AVG(score)=%f\\n\", \n        xdb_column_int (pRes->col_meta, pRow, 0), \n        xdb_column_double(pRes->col_meta, pRow, 1), \n        xdb_column_double(pRes->col_meta, pRow, 2), \n        xdb_column_double(pRes->col_meta, pRow, 3), \n        xdb_column_double(pRes->col_meta, pRow, 4));\n    printf (\"COUNT(*)=%d MIN(score)=%f MAX(score)=%f SUM(score)=%f AVG(score)=%f\\n\", \n        (int)*(int64_t*)pRow[0], \n        *(float*)pRow[1], \n        *(float*)pRow[2], \n        *(double*)pRow[3], \n        *(double*)pRow[4]);\n}\nxdb_free_result (pRes);\n
  • Transaction Rollback
printf (\"\\n=== Rollback\\n\");\nxdb_begin (pConn);\npRes = xdb_exec (pConn, \"UPDATE student set age=15 WHERE id = 2\");\npRes = xdb_exec (pConn, \"SELECT id,name,age from student WHERE id = 2\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\nprintf (\"-- rollback\\n\");\nxdb_rollback (pConn);\npRes = xdb_exec (pConn, \"SELECT id,name,age from student WHERE id = 2\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\n
  • Transaction Commit
printf (\"\\n=== Commit\\n\");\nxdb_begin (pConn);\npRes = xdb_exec (pConn, \"UPDATE student set age=15 WHERE id = 2\");\npRes = xdb_exec (pConn, \"SELECT * from student WHERE id = 2\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\nprintf (\"-- commit\\n\");\nxdb_commit (pConn);\npRes = xdb_exec (pConn, \"SELECT * from student WHERE id = 2\");\nprintf (\"select %d rows\\n\", (int)pRes->row_count);\nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\n
  • Multi-Statements
printf (\"\\n=== Muti-Statements\\n\");\npRes = xdb_exec (pConn, \"SELECT COUNT(*) FROM student; SELECT id,name FROM student WHERE id=2\");\n// count(*) \nif (NULL != (pRow = xdb_fetch_row (pRes))) {\n    xdb_print_row (pRes->col_meta, pRow, 0);\n    printf (\"\\n\");\n}\nxdb_free_result (pRes);\n// select\npRes = xdb_next_result (pConn);\nif (NULL != pRes) {\n    if (NULL != (pRow = xdb_fetch_row (pRes))) {\n        xdb_print_row (pRes->col_meta, pRow, 0);\n        printf (\"\\n\");\n    }\n    xdb_free_result (pRes);\n}\n\nxdb_exec (pConn, \"SHELL\");\n
"},{"location":"replication/logic-replication/","title":"Logical Replication","text":""},{"location":"replication/logic-replication/#create-replication","title":"Create Replication","text":"
CREATE REPLICATION rep_name [replication_option] ...\n\nreplication_option:\n    HOST = xx \n  | PORT = xx \n  | USER = xx \n  | PASSWORD = xx \n  | DB = (dbname,...)\n  | INITSYNC = {true | false}\n  | AUDIT = {true | false}\n  | SCHEMA = {true | false}\n  | ENABLE = {true | false}\n
"},{"location":"replication/logic-replication/#change-replication","title":"Change Replication","text":"
ALTER REPLICATION rep_name [replication_option] ...\n\nreplication_option:\n    ENABLE = {true | false}\n  | USER = xx \n  | PASSWORD = xx \n
"},{"location":"replication/logic-replication/#drop-replication","title":"Drop Replication","text":"
DROP REPLICATION [IF EXISTS] rep_name\n
"},{"location":"replication/logic-replication/#show-replication","title":"Show Replication","text":"
SHOW REPLICATION\n
"},{"location":"replication/pubsub/","title":"PUBSUB","text":""},{"location":"replication/pubsub/#create-subscription","title":"Create Subscription","text":"
CREATE SUBSCRIPTION sub_name [subscripton_option] ...\n\nsubscripton_option:\n    HOST = xx \n  | PORT = xx \n  | USER = xx \n  | PASSWORD = xx \n  | SRCDB = db_name\n  | DB = db_name\n  | INITSYNC = {FULL | SCHEMA | DATA | NONE}\n  | OFFSET = 0, -1\n  | AUDIT = {true | false}\n  | ENABLE = {true | false}\n\n    FILTERS (select * from xxx where xx into mytable)\n\n  | TABLES = (tbl_name,...)\n
"},{"location":"replication/pubsub/#change-replication","title":"Change Replication","text":"
ALTER REPLICATION rep_name [replication_option] ...\n\nreplication_option:\n    ENABLE = {true | false}\n  | USER = xx \n  | PASSWORD = xx \n
"},{"location":"replication/pubsub/#drop-replication","title":"Drop Replication","text":"
DROP REPLICATION [IF EXISTS] rep_name\n
"},{"location":"replication/pubsub/#show-replication","title":"Show Replication","text":"
SHOW REPLICATION\n
"},{"location":"replication/replication/","title":"Replication","text":""},{"location":"replication/replication/#create-replication","title":"Create Replication","text":"
CREATE REPLICATION rep_name [replication_option] ...\n\nreplication_option:\n    HOST = xx \n  | PORT = xx \n  | USER = xx \n  | PASSWORD = xx \n  | DB = db_name\n  | TABLES = (tbl_name,...)\n  | TODB = db_name\n  | INITSYNC = {FULL | SCHEMA | DATA | NONE}\n  | AUDIT = {true | false}\n  | ENABLE = {true | false}\n\n    FILTERS (select * from xxx where xx into mytable)\n
"},{"location":"replication/replication/#change-replication","title":"Change Replication","text":"
ALTER REPLICATION rep_name [replication_option] ...\n\nreplication_option:\n    ENABLE = {true | false}\n  | OFFSET = 0, -1\n  | USER = xx \n  | PASSWORD = xx \n
"},{"location":"replication/replication/#drop-replication","title":"Drop Replication","text":"
DROP REPLICATION [IF EXISTS] rep_name\n
"},{"location":"replication/replication/#show-replication","title":"Show Replication","text":"
SHOW REPLICSTATUS\n    show secondary status\n\nSHOW REPLICATION\n    show my replica status\n\nrole: master/slave\n\nSHOW MASTER STATUS\n\nSHOW REPLICA STATUS [FOR CHANNEL channel]\nSHOW SLAVE STATUS [FOR CHANNEL channel]\n
"},{"location":"server/server/","title":"Server","text":""},{"location":"server/server/#create-server","title":"Create Server","text":"
CREATE SERVER [IF NOT EXISTS] server_name [server_option ...]\n\nserver_option:\n    IP = ipaddr\n  | PORT = xxx \n  | TYPE = {SERVER | WEBUI | MYSQL}\n  | AUTH = {NO | PASSWORD | ACL}\n  | ENABLE = {true | false}\n  | PASSWORD = 'string'\n  | DB = (db_name,...)\n
"},{"location":"server/server/#modify-server","title":"Modify Server","text":"
ALTER SERVER server_name [server_option ...]\n\nserver_option:\n  | AUTH = {NO | PASSWORD | ACL}\n  | ENABLE = {true | false}\n  | PASSWORD = xxxx\n  | DB = (db_name,...)\n
"},{"location":"server/server/#drop-server","title":"Drop Server","text":"
DROP SERVER [IF EXISTS] server_name\n
"},{"location":"server/server/#show-server","title":"Show Server","text":"
SHOW SERVERS\n
"},{"location":"server/status/","title":"Status","text":""},{"location":"server/status/#show-status","title":"Show Status","text":"

SHOW [SESSION | GLOBAL] STATUS\n
global status, uptime, dbs, tables, sql stats

"},{"location":"server/status/#show-sessions","title":"Show Sessions","text":"

SHOW SESSIONS\n
pid, connect time, (telnet|client), ip, port, user, sql stats

"},{"location":"sql/backup/","title":"Backup and Restore","text":""},{"location":"sql/backup/#source","title":"Source","text":"

Load and execute SQL file into current database

SOURCE '[path/]sqlfile'\n
"},{"location":"sql/backup/#dump-database","title":"Dump Database","text":"

Dump current database to console

DUMP DATABASE\n

Dump specified database with options

DUMP DATABASE db_name [dump_option] ...\n\ndump_option:\n    NODROP      : Don't add DROP statement\n  | NOCREATE    : Don't add CREATE statement\n  | NODATA      : Don't dump ROW data\n  | INTO '[path/]outfile' : Dump into file instead of console\n
"},{"location":"sql/data-types/","title":"Data Types","text":"Type Bytes Description Note BOOL 1 Bool, the value range is {true, false}. TBD TINYINT 1 Single-byte integer, the value range is [-128, 127]. SMALLINT 2 Short integer, the value range is [-32768, 32767]. INT 4 Integer, the value range is [-2^31, 2^31-1]. BIGINT 8 Long integer, the value range is [-2^63, 2^63-1]. TINYINT UNSIGNED 1 unsigned single-byte integer, the value range is [0, 255]. TBD SMALLINT UNSIGNED 2 unsigned integer, the value range is [0, 65535]. TBD INT UNSIGNED 4 Unsigned integer, the value range is [0, 2^32-1]. TBD BIGINT UNSIGNED 8 unsigned long integer, the value range is [0, 2^64-1]. TBD TIMESTAMP 8 Default precision is microsecond. TBD FLOAT 4 Floating point number. DOUBLE 8 Double precision floating point number. CHAR [COLLATE collation_name] User-defined, max 65535 Fixed-length UTF-8 string. VARCHAR [COLLATE collation_name] User-defined, max 65535 Variable-Length UTF-8 string. TBD BINARY User-defined, max 65535 Fixed-length binary data. TBD VARBINARY User-defined, max 65535 Variable-Length binary data. TBD"},{"location":"sql/data-types/#collation","title":"Collation","text":"Type Description NOCASE Case insensitive [Default]. BINARY Case sensitive."},{"location":"sql/data-types/#escape-characters","title":"Escape Characters","text":"Escape Character Actual Meaning \\n Line Break \\r Carriage Return \\t tab \\' Single quote ' \\\" Double quote \" ` backtick ` \\ Back Slash \\ \\% % see below for details _ _ see below for details"},{"location":"sql/database/","title":"Database","text":""},{"location":"sql/database/#create-database","title":"Create Database","text":"
CREATE DATABASE [IF NOT EXISTS] {db_name | '[path/]db_name'} [db_option] ...\n\ndb_option:\n    ENGINE = {MMAP | MEMORY}\n  | LOCKMODE = {THREAD | PROCESS | NOLOCK}\n
LOCKMODE Lock used Description THREAD [default] high-performance thread read-write lock For single process multiple threads access. PROCESS file read-write lock For multiple processes multiple threads access. NOLOCK no lock For test/diag purpose only.

Note

'[path]/db_name' is only valid for embedded database. db_options are default values for new created tables and you can use table_options to overwrite these default values.

"},{"location":"sql/database/#open-database","title":"Open Database","text":"
OPEN DATABASE {db_name | '[path/]db_name'} [dbg_option] ...\n\ndbg_option:\n    FLUSHMODE = {ASYNC | SYNC | NOSYNC}\n  | LOCKMODE = {THREAD | PROCESS | NOLOCK}\n

Note

This command can only be used for embedded database. dbg_option is only valid for current session and only for debug purpose.

"},{"location":"sql/database/#close-database","title":"Close Database","text":"
CLOSE DATABASE [IF EXISTS] db_name\n

Note

This command can only be used for embedded database.

"},{"location":"sql/database/#drop-database","title":"Drop Database","text":"
DROP DATABASE [IF EXISTS] db_name\n
"},{"location":"sql/database/#use-database","title":"Use Database","text":"
USE db_name\n
"},{"location":"sql/database/#show-databases","title":"Show Databases","text":"
SHOW DATABASES [WHERE expr]\n
XDB> show DATABASES ;\n+----------+--------+-----------+\n| database | engine | data_path |\n+----------+--------+-----------+\n| system   |        |           |\n| memory   |        |           |\n+----------+--------+-----------+\n2 rows in set (0.012 ms)\n
"},{"location":"sql/dml/","title":"Data manipulation","text":""},{"location":"sql/dml/#insert","title":"Insert","text":"
INSERT INTO tbl_name \n    [(col_name,...)] \n    VALUES (val,...), ...\n\nINSERT INTO tbl_name SET col_name = value, [col_name = value], ...\n
"},{"location":"sql/dml/#replace","title":"Replace","text":"
REPLACE INTO tbl_name \n    [(col_name,...)] \n    VALUES (val,...), ...\n
"},{"location":"sql/dml/#select","title":"Select","text":"
SELECT col_name,... FROM tbl_name\n    [WHERE expr]\n    [ORDER BY col_name [ASC | DESC], ...]\n    [LIMIT {[offset,] row_count | row_count OFFSET offset}]\n
"},{"location":"sql/dml/#update","title":"Update","text":"
UPDATE tbl_name\n    SET col_name=val, ... \n    [WHERE expr]\n    [ORDER BY col_name [ASC | DESC], ...]\n    [LIMIT {[offset,] row_count | row_count OFFSET offset}]\n
"},{"location":"sql/dml/#delete","title":"Delete","text":"
DELETE FROM tbl_name\n    [WHERE expr]\n    [ORDER BY col_name [ASC | DESC], ...]\n    [LIMIT {[offset,] row_count | row_count OFFSET offset}]\n
"},{"location":"sql/dml/#explain","title":"Explain","text":"
EXPLAIN SELECT statement\n
"},{"location":"sql/functions/","title":"Functions","text":""},{"location":"sql/functions/#aggregate-functions","title":"Aggregate Functions","text":"Function Return Type Description COUNT(*) BIGINT The number of rows. COUNT(col_name) BIGINT The number of rows in the specified column. MIN(col_name) Same with column The minimum value of a specific column MAX(col_name) Same with column The maximum value of a specific column. SUM(col_name) DOUBLE or BIGINT The sum of a specific column in a table AVG(col_name) DOUBLE The average value of the specified column."},{"location":"sql/indexes/","title":"Index","text":""},{"location":"sql/indexes/#create-index","title":"Create Index","text":"
CREATE [UNIQUE] INDEX idx_name \n    [USING {HASH | BTREE}] \n    ON tbl_name (col_name,...)\n
"},{"location":"sql/indexes/#drop-index","title":"Drop Index","text":"
DROP INDEX idx_name ON tbl_name\n
"},{"location":"sql/indexes/#show-index","title":"Show Index","text":"
SHOW {INDEX | INDEXES | KEYS} [FROM tbl_name] [WHERE expr]\n
XDB> show INDEXES ;\n+---------+---------+------+----------+\n| table   | idx_key | type | col_list |\n+---------+---------+------+----------+\n| student | PRIMARY | HASH | id       |\n| student | name_2  | HASH | name     |\n| teacher | PRIMARY | HASH | id       |\n| teacher | name_2  | HASH | name     |\n| book    | PRIMARY | HASH | id       |\n| book    | name_2  | HASH | name     |\n+---------+---------+------+----------+\n6 rows in set (0.016 ms)\n
"},{"location":"sql/misc/","title":"Misc","text":""},{"location":"sql/misc/#shell","title":"Shell","text":"

Enter interactive shell

SHELL\n
"},{"location":"sql/misc/#help","title":"Help","text":"
HELP [topic] [subtopic]\n
"},{"location":"sql/name-limit/","title":"Name and Limit","text":""},{"location":"sql/name-limit/#case-sensitive","title":"Case Sensitive","text":"Type Case-sensitive Keyword False Function False DB Name False Table Name False Column Name False Index Name False"},{"location":"sql/name-limit/#limit","title":"Limit","text":"Type Limit DB Name 64 Table Name 64 Column Name 64 Index Name 64 Single SQL statement 1048576 (1M) DB per process 1024 Table per DB 4095 Rows per Table 2,000,000,000 (2B) Row size 524288 (512K) Index per Table 16 Filter Fields 64"},{"location":"sql/operators/","title":"Operators","text":""},{"location":"sql/operators/#comparison-operators","title":"Comparison Operators","text":"Operator Description Note = Equal to <>, != Not equal to TBD > Greater than TBD < Less than TBD >= Greater than or equal to TBD <= Less than or equal to TBD IS NULL NULL value test TBD IS NOT NULL NOT NULL value test TBD BETWEEN ... AND ... Whether a value is within a range of values TBD NOT BETWEEN ... AND ... Whether a value is not within a range of values TBD IN Whether a value is not within a set of values TBD NOT IN Negation of simple pattern matching TBD LIKE Wildcard match TBD

LIKE is used together with wildcards to match strings. Its usage is described as follows:

  • '%' matches 0 or any number of characters, '_' matches any single ASCII character.
  • \\_ is used to match the _ in the string.
"},{"location":"sql/operators/#logical-operators","title":"Logical Operators","text":"Operator Description Note AND Logical AND OR Logical OR TBD"},{"location":"sql/statements/","title":"SQL Statements","text":"Statement Descritpion Note BEGIN Start a new transaction. COMMIT Commits the open transaction if exists. CLOSE DATABASE Close an opened database. Embedded Mode only CREATE DATABASE Create a database. CREATE INDEX Create an index on the specified table. CREATE TABLE Create a new table. DELETE Delete rows in a table. DESCRIBE Describe the specified table. DROP DATABASE Drop a database. DROP INDEX Drop the specified index on the specified table. DROP TABLE Drop the specified table. DUMP DATABASE Dump database into file. EXPLAIN Returns detailed information on how the query is executed. HELP SQL statements help. INSERT Insert data into a table. OPEN DATABASE Open a database on disk. Embedded Mode only ROLLBACK Roll back the open transaction if exists. SELECT Retrieves data from a table. SET AUTOCOMMIT Set session transaction auto-commit. SHELL Enter interactive shell. SHOW COLUMNS Show the list of columns within a given table. SHOW CREATE TABLE Show the CREATE TABLE statement. SHOW DATABASE Show the list of databases. SHOW INDEXES Show the list of indexes associated with a given table. SHOW TABLES Show the list of tables in database. SOURCE Load and execute SQL file. UPDATE Modifies rows in a table. USE Use the database as the current default database."},{"location":"sql/table/","title":"Table","text":""},{"location":"sql/table/#create-table","title":"Create Table","text":"
CREATE TABLE [IF NOT EXISTS] tbl_name (create_definition, ...) [table_option] ...\n\nCREATE OR REPLACE TABLE tbl_name (create_definition, ...)  [table_option] ...\n\ncreate_definition:\n    col_name data_type [column_option] ...\n  | PRIMARY KEY index_definition\n  | UNIQUE [KEY | INDEX] [idx_name] index_definition\n  | {KEY | INDEX} [idx_name] index_definition\n\ndata_type:\n    BOOL\n  | TINYINT [UNSIGNED]\n  | SMALLINT [UNSIGNED]\n  | {INT | INTEGER} [UNSIGNED]\n  | BIGINT [UNSIGNED]\n  | TIMESTAMP\n  | CHAR[(n)] [COLLATE <collation_name>]\n  | VARCHAR[(n)] [COLLATE <collation_name>]\n  | BINARY[(n)]\n  | VBINARY[(n)]\n\ncollation_name:\n    BINARY\n  | NOCASE\n\ncolumn_option:\n    {NOT NULL | NULL} \n  | DEFAULT value\n  | AUTO_INCREMENT \n  | [PRIMARY] KEY\n  | UNIQUE [KEY]\n\nindex_definition: [USING {HASH | BTREE}] (col_name, ...)\n\ntable_option:\n    ENGINE = MEMORY\n  | MAX_ROWS = value\n  | MIN_ROWS = value\n  | LOCKMODE = {THREAD | PROCESS | NOLOCK}\n  | ROWLOCK = {0 | 1}\n  | MVCC = {0 | 1}\n
"},{"location":"sql/table/#drop-table","title":"Drop Table","text":"
DROP TABLE [IF EXISTS] tbl_name\n
"},{"location":"sql/table/#show-tables","title":"Show Tables","text":""},{"location":"sql/table/#show-all-tables","title":"Show All Tables","text":"
SHOW TABLES [LIKE pattern] [WHERE expr]\n
XDB> show TABLES ;\n+---------+--------+-------------+-----------+\n| table   | engine | primary_key | data_path |\n+---------+--------+-------------+-----------+\n| student |        |             |           |\n| teacher |        |             |           |\n| book    |        |             |           |\n+---------+--------+-------------+-----------+\n3 rows in set (0.022 ms)\n
"},{"location":"sql/table/#show-create-table","title":"Show Create Table","text":"
SHOW CREATE TABLE tbl_name\n
XDB> show CREATE TABLE student ;\n+-------------------------------+\n| schema                        |\n+-------------------------------+\n| CREATE TABLE student (        |\n|   id               INT,       |\n|   name             CHAR(16),  |\n|   age              INT,       |\n|   class            CHAR(16),  |\n|   score            FLOAT,     |\n|   info             CHAR(256), |\n|   PRIMARY KEY (id),           |\n|   KEY         name_2 (name)   |\n| );                            |\n+-------------------------------+\n1 row in set (0.026 ms)\n
"},{"location":"sql/table/#show-table-columns","title":"Show Table Columns","text":"
{DESC | DESCRIBE} tbl_name\n\nSHOW COLUMNS [FROM tbl_name] [WHERE expr]\n
XDB> DESCRIBE student ;\n+--------+-------+-----+\n| column | type  | len |\n+--------+-------+-----+\n| id     | INT   | 4   |\n| name   | CHAR  | 16  |\n| age    | INT   | 4   |\n| class  | CHAR  | 16  |\n| score  | FLOAT | 4   |\n| info   | CHAR  | 256 |\n+--------+-------+-----+\n6 rows in set (0.023 ms)\n
"},{"location":"sql/transaction/","title":"Transaction and Lock","text":""},{"location":"sql/transaction/#begin-transaction","title":"Begin Transaction","text":"
BEGIN\n\nSTART TRANSACTION [READ WRITE | READ ONLY]\n

Note

Will commit any existing open transaction

"},{"location":"sql/transaction/#commit-transaction","title":"Commit Transaction","text":"
COMMIT\n
"},{"location":"sql/transaction/#rollback-transaction","title":"Rollback Transaction","text":"
ROLLBACK\n
"},{"location":"sql/transaction/#lock-tables","title":"Lock Tables","text":"
-- TBD\nLOCK {TABLE | TABLES} tbl_name { READ | WRITE } [, tbl_name { READ | WRITE }] ...\n

Note

This command must be executed after BEGIN transaction for performance or prevent other sessions to , and locks will be released after commit transaction.

"},{"location":"sql/transaction/#set-isolation-level","title":"Set Isolation Level","text":"
-- TBD\nSET TRANSACTION ISOLATION LEVEL { READ COMMITTED | REPEATABLE READ | READ UNCOMMITTED | SERIALIZABLE }\n

Note

TBD, default is READ COMMITTED.

"},{"location":"sql/transaction/#set-auto-commit","title":"Set Auto Commit","text":"
SET AUTOCOMMIT = {0 | 1}\n

Note

Default AUTOCOMMIT = 1.

"},{"location":"sysdb/information-schema/","title":"INFORMATION_SCHEMA","text":"

TBD

"},{"location":"sysdb/system/","title":"system","text":""},{"location":"sysdb/system/#databases","title":"Databases","text":"
XDB> SELECT * FROM system.databases;\n+----------+--------+-----------+\n| database | engine | data_path |\n+----------+--------+-----------+\n| system   |        |           |\n| memory   |        |           |\n+----------+--------+-----------+\n2 rows in set (0.010 ms)\n
"},{"location":"sysdb/system/#tables","title":"Tables","text":"
XDB> SELECT * FROM system.tables where database='memory';\n+----------+---------+--------+-----------+-------------------------------+\n| database | table   | engine | data_path | schema                        |\n+----------+---------+--------+-----------+-------------------------------+\n| memory   | student |        |           | CREATE TABLE student (        |\n|          |         |        |           |   id               INT,       |\n|          |         |        |           |   name             CHAR(16),  |\n|          |         |        |           |   age              INT,       |\n|          |         |        |           |   class            CHAR(16),  |\n|          |         |        |           |   score            FLOAT,     |\n|          |         |        |           |   info             CHAR(256), |\n|          |         |        |           |   PRIMARY KEY (id),           |\n|          |         |        |           |   KEY         name_2 (name)   |\n|          |         |        |           | );                            |\n| memory   | teacher |        |           | CREATE TABLE teacher (        |\n|          |         |        |           |   id               INT,       |\n|          |         |        |           |   name             CHAR(16),  |\n|          |         |        |           |   age              INT,       |\n|          |         |        |           |   info             CHAR(256), |\n|          |         |        |           |   PRIMARY KEY (id),           |\n|          |         |        |           |   KEY         name_2 (name)   |\n|          |         |        |           | );                            |\n| memory   | book    |        |           | CREATE TABLE book (           |\n|          |         |        |           |   id               INT,       |\n|          |         |        |           |   name             CHAR(64),  |\n|          |         |        |           |   author           CHAR(32),  |\n|          |         |        |           |   count            INT,       |\n|          |         |        |           |   PRIMARY KEY (id),           |\n|          |         |        |           |   KEY         name_2 (name)   |\n|          |         |        |           | );                            |\n+----------+---------+--------+-----------+-------------------------------+\n3 rows in set (0.020 ms)\n
"},{"location":"sysdb/system/#columns","title":"Columns","text":"
XDB> SELECT * FROM system.columns where database='memory';\n+----------+---------+--------+-------+-----+\n| database | table   | column | type  | len |\n+----------+---------+--------+-------+-----+\n| memory   | student | id     | INT   | 4   |\n| memory   | student | name   | CHAR  | 16  |\n| memory   | student | age    | INT   | 4   |\n| memory   | student | class  | CHAR  | 16  |\n| memory   | student | score  | FLOAT | 4   |\n| memory   | student | info   | CHAR  | 256 |\n| memory   | teacher | id     | INT   | 4   |\n| memory   | teacher | name   | CHAR  | 16  |\n| memory   | teacher | age    | INT   | 4   |\n| memory   | teacher | info   | CHAR  | 256 |\n| memory   | book    | id     | INT   | 4   |\n| memory   | book    | name   | CHAR  | 64  |\n| memory   | book    | author | CHAR  | 32  |\n| memory   | book    | count  | INT   | 4   |\n+----------+---------+--------+-------+-----+\n14 rows in set (0.017 ms)\n
"},{"location":"sysdb/system/#indexes","title":"Indexes","text":"
XDB> SELECT * FROM system.indexes where database='memory';\n+----------+---------+---------+------+----------+\n| database | table   | idx_key | type | col_list |\n+----------+---------+---------+------+----------+\n| memory   | student | PRIMARY | HASH | id       |\n| memory   | student | name_2  | HASH | name     |\n| memory   | teacher | PRIMARY | HASH | id       |\n| memory   | teacher | name_2  | HASH | name     |\n| memory   | book    | PRIMARY | HASH | id       |\n| memory   | book    | name_2  | HASH | name     |\n+----------+---------+---------+------+----------+\n6 rows in set (0.016 ms)\n
"}]} \ No newline at end of file diff --git a/server/server/index.html b/server/server/index.html new file mode 100755 index 0000000..346daf8 --- /dev/null +++ b/server/server/index.html @@ -0,0 +1,50 @@ + Server - CrossDB

Server

Create Server

CREATE SERVER [IF NOT EXISTS] server_name [server_option ...]
+
+server_option:
+    IP = ipaddr
+  | PORT = xxx 
+  | TYPE = {SERVER | WEBUI | MYSQL}
+  | AUTH = {NO | PASSWORD | ACL}
+  | ENABLE = {true | false}
+  | PASSWORD = 'string'
+  | DB = (db_name,...)
+

Modify Server

ALTER SERVER server_name [server_option ...]
+
+server_option:
+  | AUTH = {NO | PASSWORD | ACL}
+  | ENABLE = {true | false}
+  | PASSWORD = xxxx
+  | DB = (db_name,...)
+

Drop Server

DROP SERVER [IF EXISTS] server_name
+

Show Server

SHOW SERVERS
+

Comments

\ No newline at end of file diff --git a/server/status/index.html b/server/status/index.html new file mode 100755 index 0000000..709c845 --- /dev/null +++ b/server/status/index.html @@ -0,0 +1,33 @@ + Status - CrossDB

Status

Show Status

SHOW [SESSION | GLOBAL] STATUS
+
global status, uptime, dbs, tables, sql stats

Show Sessions

SHOW SESSIONS
+
pid, connect time, (telnet|client), ip, port, user, sql stats

Comments

\ No newline at end of file diff --git a/sitemap.xml b/sitemap.xml old mode 100644 new mode 100755 index 174bc69..8796c0c --- a/sitemap.xml +++ b/sitemap.xml @@ -2,202 +2,222 @@ https://crossdb.org/ - 2024-03-01 + 2024-08-18 daily - https://crossdb.org/about/about/ - 2024-03-01 + https://crossdb.org/CHANGELOG/ + 2024-08-18 daily - https://crossdb.org/about/buy/ - 2024-03-01 + https://crossdb.org/community/ + 2024-08-18 + daily + + + https://crossdb.org/faq/ + 2024-08-18 + daily + + + https://crossdb.org/glossary/ + 2024-08-18 daily - https://crossdb.org/about/community/ - 2024-03-01 + https://crossdb.org/intro/ + 2024-08-18 daily - https://crossdb.org/about/faq/ - 2024-03-01 + https://crossdb.org/about/buy/ + 2024-08-18 daily https://crossdb.org/about/privacy/ - 2024-03-01 + 2024-08-18 daily https://crossdb.org/about/refund/ - 2024-03-01 + 2024-08-18 daily https://crossdb.org/about/terms/ - 2024-03-01 + 2024-08-18 + daily + + + https://crossdb.org/admin/data-backup/ + 2024-08-18 + daily + + + https://crossdb.org/admin/data-restore/ + 2024-08-18 daily - https://crossdb.org/backup/ - 2024-03-01 + https://crossdb.org/admin/shell/ + 2024-08-18 daily - https://crossdb.org/backup/features/ - 2024-03-01 + https://crossdb.org/client/api-c/ + 2024-08-18 daily - https://crossdb.org/backup/introduction/ - 2024-03-01 + https://crossdb.org/client/api-python/ + 2024-08-18 daily - https://crossdb.org/backup/usecases/ - 2024-03-01 + https://crossdb.org/develop/connect/ + 2024-08-18 daily - https://crossdb.org/backup/blog/avx512-in-golang/ - 2024-03-01 + https://crossdb.org/develop/ddl/ + 2024-08-18 daily - https://crossdb.org/backup/blog/frontend-in-go-modules/ - 2024-03-01 + https://crossdb.org/develop/dml/ + 2024-08-18 daily - https://crossdb.org/backup/reference/crossline/ - 2024-03-01 + https://crossdb.org/develop/multi-stmts/ + 2024-08-18 daily - https://crossdb.org/backup/reference/markdown/ - 2024-03-01 + https://crossdb.org/develop/prep-stmts/ + 2024-08-18 daily - https://crossdb.org/blog/why-new-embdb/ - 2024-03-01 + https://crossdb.org/develop/query/ + 2024-08-18 daily - https://crossdb.org/blog/benchmark/crossdb-vs-lmdb/ - 2024-03-01 + https://crossdb.org/develop/transaction/ + 2024-08-18 daily - https://crossdb.org/blog/benchmark/crossdb-vs-sqlite3/ - 2024-03-01 + https://crossdb.org/get-started/bench/ + 2024-08-18 daily - https://crossdb.org/blog/benchmark/crossdb-vs-stlhmap/ - 2024-03-01 + https://crossdb.org/get-started/install/ + 2024-08-18 daily - https://crossdb.org/blog/benchmark/crossdb-vs-stlmap/ - 2024-03-01 + https://crossdb.org/get-started/tutorial/ + 2024-08-18 daily - https://crossdb.org/docs/get-started/ - 2024-03-01 + https://crossdb.org/replication/logic-replication/ + 2024-08-18 daily - https://crossdb.org/docs/introduction/ - 2024-03-01 + https://crossdb.org/replication/pubsub/ + 2024-08-18 daily - https://crossdb.org/docs/api/api/ - 2024-03-01 + https://crossdb.org/replication/replication/ + 2024-08-18 daily - https://crossdb.org/docs/api/ddl/ - 2024-03-01 + https://crossdb.org/server/server/ + 2024-08-18 daily - https://crossdb.org/docs/api/dml/ - 2024-03-01 + https://crossdb.org/server/status/ + 2024-08-18 daily - https://crossdb.org/docs/api/misc/ - 2024-03-01 + https://crossdb.org/sql/backup/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/database/ - 2024-03-01 + https://crossdb.org/sql/data-types/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/dml/ - 2024-03-01 + https://crossdb.org/sql/database/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/indexes/ - 2024-03-01 + https://crossdb.org/sql/dml/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/schema/ - 2024-03-01 + https://crossdb.org/sql/functions/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/table/ - 2024-03-01 + https://crossdb.org/sql/indexes/ + 2024-08-18 daily - https://crossdb.org/docs/crossdb/transaction/ - 2024-03-01 + https://crossdb.org/sql/misc/ + 2024-08-18 daily - https://crossdb.org/docs/reference/crossbench/ - 2024-03-01 + https://crossdb.org/sql/name-limit/ + 2024-08-18 daily - https://crossdb.org/docs/reference/crossdb-cli/ - 2024-03-01 + https://crossdb.org/sql/operators/ + 2024-08-18 daily - https://crossdb.org/products/CHANGELOG/ - 2024-03-01 + https://crossdb.org/sql/statements/ + 2024-08-18 daily - https://crossdb.org/products/download/ - 2024-03-01 + https://crossdb.org/sql/table/ + 2024-08-18 daily - https://crossdb.org/products/price/ - 2024-03-01 + https://crossdb.org/sql/transaction/ + 2024-08-18 daily - https://crossdb.org/products/products/ - 2024-03-01 + https://crossdb.org/sysdb/information-schema/ + 2024-08-18 daily - https://crossdb.org/products/sponsor/ - 2024-03-01 + https://crossdb.org/sysdb/system/ + 2024-08-18 daily \ No newline at end of file diff --git a/sitemap.xml.gz b/sitemap.xml.gz old mode 100644 new mode 100755 index 70e7cac..703fbe4 Binary files a/sitemap.xml.gz and b/sitemap.xml.gz differ diff --git a/sql/backup/index.html b/sql/backup/index.html new file mode 100755 index 0000000..2cfad18 --- /dev/null +++ b/sql/backup/index.html @@ -0,0 +1,68 @@ + Backup and Restore - CrossDB

Backup and Restore

Source

Load and execute SQL file into current database

SOURCE '[path/]sqlfile'
+

Dump Database

Dump current database to console

DUMP DATABASE
+

Dump specified database with options

DUMP DATABASE db_name [dump_option] ...
+
+dump_option:
+    NODROP      : Don't add DROP statement
+  | NOCREATE    : Don't add CREATE statement
+  | NODATA      : Don't dump ROW data
+  | INTO '[path/]outfile' : Dump into file instead of console
+

Comments

\ No newline at end of file diff --git a/sql/data-types/index.html b/sql/data-types/index.html new file mode 100755 index 0000000..a6dd607 --- /dev/null +++ b/sql/data-types/index.html @@ -0,0 +1,31 @@ + Data Types - CrossDB

Data Types

Type Bytes Description Note
BOOL 1 Bool, the value range is {true, false}. TBD
TINYINT 1 Single-byte integer, the value range is [-128, 127].
SMALLINT 2 Short integer, the value range is [-32768, 32767].
INT 4 Integer, the value range is [-2^31, 2^31-1].
BIGINT 8 Long integer, the value range is [-2^63, 2^63-1].
TINYINT UNSIGNED 1 unsigned single-byte integer, the value range is [0, 255]. TBD
SMALLINT UNSIGNED 2 unsigned integer, the value range is [0, 65535]. TBD
INT UNSIGNED 4 Unsigned integer, the value range is [0, 2^32-1]. TBD
BIGINT UNSIGNED 8 unsigned long integer, the value range is [0, 2^64-1]. TBD
TIMESTAMP 8 Default precision is microsecond. TBD
FLOAT 4 Floating point number.
DOUBLE 8 Double precision floating point number.
CHAR [COLLATE collation_name] User-defined, max 65535 Fixed-length UTF-8 string.
VARCHAR [COLLATE collation_name] User-defined, max 65535 Variable-Length UTF-8 string. TBD
BINARY User-defined, max 65535 Fixed-length binary data. TBD
VARBINARY User-defined, max 65535 Variable-Length binary data. TBD

Collation

Type Description
NOCASE Case insensitive [Default].
BINARY Case sensitive.

Escape Characters

Escape Character Actual Meaning
\n Line Break
\r Carriage Return
\t tab
\' Single quote '
\" Double quote "
` backtick `
\ Back Slash \
\% % see below for details
_ _ see below for details

Comments

\ No newline at end of file diff --git a/sql/database/index.html b/sql/database/index.html new file mode 100755 index 0000000..bda89b7 --- /dev/null +++ b/sql/database/index.html @@ -0,0 +1,78 @@ + Database - CrossDB

Database

Create Database

CREATE DATABASE [IF NOT EXISTS] {db_name | '[path/]db_name'} [db_option] ...
+
+db_option:
+    ENGINE = {MMAP | MEMORY}
+  | LOCKMODE = {THREAD | PROCESS | NOLOCK}
+
LOCKMODE Lock used Description
THREAD [default] high-performance thread read-write lock For single process multiple threads access.
PROCESS file read-write lock For multiple processes multiple threads access.
NOLOCK no lock For test/diag purpose only.

Note

'[path]/db_name' is only valid for embedded database. db_options are default values for new created tables and you can use table_options to overwrite these default values.

Open Database

OPEN DATABASE {db_name | '[path/]db_name'} [dbg_option] ...
+
+dbg_option:
+    FLUSHMODE = {ASYNC | SYNC | NOSYNC}
+  | LOCKMODE = {THREAD | PROCESS | NOLOCK}
+

Note

This command can only be used for embedded database. dbg_option is only valid for current session and only for debug purpose.

Close Database

CLOSE DATABASE [IF EXISTS] db_name
+

Note

This command can only be used for embedded database.

Drop Database

DROP DATABASE [IF EXISTS] db_name
+

Use Database

USE db_name
+

Show Databases

SHOW DATABASES [WHERE expr]
+
XDB> show DATABASES ;
++----------+--------+-----------+
+| database | engine | data_path |
++----------+--------+-----------+
+| system   |        |           |
+| memory   |        |           |
++----------+--------+-----------+
+2 rows in set (0.012 ms)
+

Comments

\ No newline at end of file diff --git a/sql/dml/index.html b/sql/dml/index.html new file mode 100755 index 0000000..2a78d85 --- /dev/null +++ b/sql/dml/index.html @@ -0,0 +1,72 @@ + Data manipulation - CrossDB

Data manipulation

Insert

INSERT INTO tbl_name 
+    [(col_name,...)] 
+    VALUES (val,...), ...
+
+INSERT INTO tbl_name SET col_name = value, [col_name = value], ...
+

Replace

REPLACE INTO tbl_name 
+    [(col_name,...)] 
+    VALUES (val,...), ...
+

Select

SELECT col_name,... FROM tbl_name
+    [WHERE expr]
+    [ORDER BY col_name [ASC | DESC], ...]
+    [LIMIT {[offset,] row_count | row_count OFFSET offset}]
+

Update

UPDATE tbl_name
+    SET col_name=val, ... 
+    [WHERE expr]
+    [ORDER BY col_name [ASC | DESC], ...]
+    [LIMIT {[offset,] row_count | row_count OFFSET offset}]
+

Delete

DELETE FROM tbl_name
+    [WHERE expr]
+    [ORDER BY col_name [ASC | DESC], ...]
+    [LIMIT {[offset,] row_count | row_count OFFSET offset}]
+

Explain

EXPLAIN SELECT statement
+

Comments

\ No newline at end of file diff --git a/sql/functions/index.html b/sql/functions/index.html new file mode 100755 index 0000000..f30045b --- /dev/null +++ b/sql/functions/index.html @@ -0,0 +1,31 @@ + Functions - CrossDB

Functions

Aggregate Functions

Function Return Type Description
COUNT(*) BIGINT The number of rows.
COUNT(col_name) BIGINT The number of rows in the specified column.
MIN(col_name) Same with column The minimum value of a specific column
MAX(col_name) Same with column The maximum value of a specific column.
SUM(col_name) DOUBLE or BIGINT The sum of a specific column in a table
AVG(col_name) DOUBLE The average value of the specified column.

Comments

\ No newline at end of file diff --git a/sql/indexes/index.html b/sql/indexes/index.html new file mode 100755 index 0000000..965dee7 --- /dev/null +++ b/sql/indexes/index.html @@ -0,0 +1,58 @@ + Index - CrossDB

Index

Create Index

CREATE [UNIQUE] INDEX idx_name 
+    [USING {HASH | BTREE}] 
+    ON tbl_name (col_name,...)
+

Drop Index

DROP INDEX idx_name ON tbl_name
+

Show Index

SHOW {INDEX | INDEXES | KEYS} [FROM tbl_name] [WHERE expr]
+
XDB> show INDEXES ;
++---------+---------+------+----------+
+| table   | idx_key | type | col_list |
++---------+---------+------+----------+
+| student | PRIMARY | HASH | id       |
+| student | name_2  | HASH | name     |
+| teacher | PRIMARY | HASH | id       |
+| teacher | name_2  | HASH | name     |
+| book    | PRIMARY | HASH | id       |
+| book    | name_2  | HASH | name     |
++---------+---------+------+----------+
+6 rows in set (0.016 ms)
+

Comments

\ No newline at end of file diff --git a/sql/misc/index.html b/sql/misc/index.html new file mode 100755 index 0000000..0e42465 --- /dev/null +++ b/sql/misc/index.html @@ -0,0 +1,33 @@ + Misc - CrossDB

Misc

Shell

Enter interactive shell

SHELL
+

Help

HELP [topic] [subtopic]
+

Comments

\ No newline at end of file diff --git a/sql/name-limit/index.html b/sql/name-limit/index.html new file mode 100755 index 0000000..1eb74db --- /dev/null +++ b/sql/name-limit/index.html @@ -0,0 +1,31 @@ + Name and Limit - CrossDB

Name and Limit

Case Sensitive

Type Case-sensitive
Keyword False
Function False
DB Name False
Table Name False
Column Name False
Index Name False

Limit

Type Limit
DB Name 64
Table Name 64
Column Name 64
Index Name 64
Single SQL statement 1048576 (1M)
DB per process 1024
Table per DB 4095
Rows per Table 2,000,000,000 (2B)
Row size 524288 (512K)
Index per Table 16
Filter Fields 64

Comments

\ No newline at end of file diff --git a/sql/operators/index.html b/sql/operators/index.html new file mode 100755 index 0000000..4f2409e --- /dev/null +++ b/sql/operators/index.html @@ -0,0 +1,31 @@ + Operators - CrossDB

Operators

Comparison Operators

Operator Description Note
= Equal to
<>, != Not equal to TBD
> Greater than TBD
< Less than TBD
>= Greater than or equal to TBD
<= Less than or equal to TBD
IS NULL NULL value test TBD
IS NOT NULL NOT NULL value test TBD
BETWEEN ... AND ... Whether a value is within a range of values TBD
NOT BETWEEN ... AND ... Whether a value is not within a range of values TBD
IN Whether a value is not within a set of values TBD
NOT IN Negation of simple pattern matching TBD
LIKE Wildcard match TBD

LIKE is used together with wildcards to match strings. Its usage is described as follows:

  • '%' matches 0 or any number of characters, '_' matches any single ASCII character.
  • \_ is used to match the _ in the string.

Logical Operators

Operator Description Note
AND Logical AND
OR Logical OR TBD

Comments

\ No newline at end of file diff --git a/sql/statements/index.html b/sql/statements/index.html new file mode 100755 index 0000000..2424924 --- /dev/null +++ b/sql/statements/index.html @@ -0,0 +1,50 @@ + SQL Statements - CrossDB

SQL Statements

Statement Descritpion Note
BEGIN Start a new transaction.
COMMIT Commits the open transaction if exists.
CLOSE DATABASE Close an opened database. Embedded Mode only
CREATE DATABASE Create a database.
CREATE INDEX Create an index on the specified table.
CREATE TABLE Create a new table.
DELETE Delete rows in a table.
DESCRIBE Describe the specified table.
DROP DATABASE Drop a database.
DROP INDEX Drop the specified index on the specified table.
DROP TABLE Drop the specified table.
DUMP DATABASE Dump database into file.
EXPLAIN Returns detailed information on how the query is executed.
HELP SQL statements help.
INSERT Insert data into a table.
OPEN DATABASE Open a database on disk. Embedded Mode only
ROLLBACK Roll back the open transaction if exists.
SELECT Retrieves data from a table.
SET AUTOCOMMIT Set session transaction auto-commit.
SHELL Enter interactive shell.
SHOW COLUMNS Show the list of columns within a given table.
SHOW CREATE TABLE Show the CREATE TABLE statement.
SHOW DATABASE Show the list of databases.
SHOW INDEXES Show the list of indexes associated with a given table.
SHOW TABLES Show the list of tables in database.
SOURCE Load and execute SQL file.
UPDATE Modifies rows in a table.
USE Use the database as the current default database.

Comments

\ No newline at end of file diff --git a/sql/table/index.html b/sql/table/index.html new file mode 100755 index 0000000..d42f347 --- /dev/null +++ b/sql/table/index.html @@ -0,0 +1,143 @@ + Table - CrossDB

Table

Create Table

CREATE TABLE [IF NOT EXISTS] tbl_name (create_definition, ...) [table_option] ...
+
+CREATE OR REPLACE TABLE tbl_name (create_definition, ...)  [table_option] ...
+
+create_definition:
+    col_name data_type [column_option] ...
+  | PRIMARY KEY index_definition
+  | UNIQUE [KEY | INDEX] [idx_name] index_definition
+  | {KEY | INDEX} [idx_name] index_definition
+
+data_type:
+    BOOL
+  | TINYINT [UNSIGNED]
+  | SMALLINT [UNSIGNED]
+  | {INT | INTEGER} [UNSIGNED]
+  | BIGINT [UNSIGNED]
+  | TIMESTAMP
+  | CHAR[(n)] [COLLATE <collation_name>]
+  | VARCHAR[(n)] [COLLATE <collation_name>]
+  | BINARY[(n)]
+  | VBINARY[(n)]
+
+collation_name:
+    BINARY
+  | NOCASE
+
+column_option:
+    {NOT NULL | NULL} 
+  | DEFAULT value
+  | AUTO_INCREMENT 
+  | [PRIMARY] KEY
+  | UNIQUE [KEY]
+
+index_definition: [USING {HASH | BTREE}] (col_name, ...)
+
+table_option:
+    ENGINE = MEMORY
+  | MAX_ROWS = value
+  | MIN_ROWS = value
+  | LOCKMODE = {THREAD | PROCESS | NOLOCK}
+  | ROWLOCK = {0 | 1}
+  | MVCC = {0 | 1}
+

Drop Table

DROP TABLE [IF EXISTS] tbl_name
+

Show Tables

Show All Tables

SHOW TABLES [LIKE pattern] [WHERE expr]
+
XDB> show TABLES ;
++---------+--------+-------------+-----------+
+| table   | engine | primary_key | data_path |
++---------+--------+-------------+-----------+
+| student |        |             |           |
+| teacher |        |             |           |
+| book    |        |             |           |
++---------+--------+-------------+-----------+
+3 rows in set (0.022 ms)
+

Show Create Table

SHOW CREATE TABLE tbl_name
+
XDB> show CREATE TABLE student ;
++-------------------------------+
+| schema                        |
++-------------------------------+
+| CREATE TABLE student (        |
+|   id               INT,       |
+|   name             CHAR(16),  |
+|   age              INT,       |
+|   class            CHAR(16),  |
+|   score            FLOAT,     |
+|   info             CHAR(256), |
+|   PRIMARY KEY (id),           |
+|   KEY         name_2 (name)   |
+| );                            |
++-------------------------------+
+1 row in set (0.026 ms)
+

Show Table Columns

{DESC | DESCRIBE} tbl_name
+
+SHOW COLUMNS [FROM tbl_name] [WHERE expr]
+
XDB> DESCRIBE student ;
++--------+-------+-----+
+| column | type  | len |
++--------+-------+-----+
+| id     | INT   | 4   |
+| name   | CHAR  | 16  |
+| age    | INT   | 4   |
+| class  | CHAR  | 16  |
+| score  | FLOAT | 4   |
+| info   | CHAR  | 256 |
++--------+-------+-----+
+6 rows in set (0.023 ms)
+

Comments

\ No newline at end of file diff --git a/sql/transaction/index.html b/sql/transaction/index.html new file mode 100755 index 0000000..3af1cab --- /dev/null +++ b/sql/transaction/index.html @@ -0,0 +1,41 @@ + Transaction and Lock - CrossDB

Transaction and Lock

Begin Transaction

BEGIN
+
+START TRANSACTION [READ WRITE | READ ONLY]
+

Note

Will commit any existing open transaction

Commit Transaction

COMMIT
+

Rollback Transaction

ROLLBACK
+

Lock Tables

-- TBD
+LOCK {TABLE | TABLES} tbl_name { READ | WRITE } [, tbl_name { READ | WRITE }] ...
+

Note

This command must be executed after BEGIN transaction for performance or prevent other sessions to , and locks will be released after commit transaction.

Set Isolation Level

-- TBD
+SET TRANSACTION ISOLATION LEVEL { READ COMMITTED | REPEATABLE READ | READ UNCOMMITTED | SERIALIZABLE }
+

Note

TBD, default is READ COMMITTED.

Set Auto Commit

SET AUTOCOMMIT = {0 | 1}
+

Note

Default AUTOCOMMIT = 1.

Comments

\ No newline at end of file diff --git a/sysdb/information-schema/index.html b/sysdb/information-schema/index.html new file mode 100755 index 0000000..a850197 --- /dev/null +++ b/sysdb/information-schema/index.html @@ -0,0 +1,86 @@ + INFORMATION_SCHEMA - CrossDB

INFORMATION_SCHEMA

TBD

Comments

\ No newline at end of file diff --git a/sysdb/system/index.html b/sysdb/system/index.html new file mode 100755 index 0000000..e52acf4 --- /dev/null +++ b/sysdb/system/index.html @@ -0,0 +1,138 @@ + system - CrossDB

system

Databases

XDB> SELECT * FROM system.databases;
++----------+--------+-----------+
+| database | engine | data_path |
++----------+--------+-----------+
+| system   |        |           |
+| memory   |        |           |
++----------+--------+-----------+
+2 rows in set (0.010 ms)
+

Tables

XDB> SELECT * FROM system.tables where database='memory';
++----------+---------+--------+-----------+-------------------------------+
+| database | table   | engine | data_path | schema                        |
++----------+---------+--------+-----------+-------------------------------+
+| memory   | student |        |           | CREATE TABLE student (        |
+|          |         |        |           |   id               INT,       |
+|          |         |        |           |   name             CHAR(16),  |
+|          |         |        |           |   age              INT,       |
+|          |         |        |           |   class            CHAR(16),  |
+|          |         |        |           |   score            FLOAT,     |
+|          |         |        |           |   info             CHAR(256), |
+|          |         |        |           |   PRIMARY KEY (id),           |
+|          |         |        |           |   KEY         name_2 (name)   |
+|          |         |        |           | );                            |
+| memory   | teacher |        |           | CREATE TABLE teacher (        |
+|          |         |        |           |   id               INT,       |
+|          |         |        |           |   name             CHAR(16),  |
+|          |         |        |           |   age              INT,       |
+|          |         |        |           |   info             CHAR(256), |
+|          |         |        |           |   PRIMARY KEY (id),           |
+|          |         |        |           |   KEY         name_2 (name)   |
+|          |         |        |           | );                            |
+| memory   | book    |        |           | CREATE TABLE book (           |
+|          |         |        |           |   id               INT,       |
+|          |         |        |           |   name             CHAR(64),  |
+|          |         |        |           |   author           CHAR(32),  |
+|          |         |        |           |   count            INT,       |
+|          |         |        |           |   PRIMARY KEY (id),           |
+|          |         |        |           |   KEY         name_2 (name)   |
+|          |         |        |           | );                            |
++----------+---------+--------+-----------+-------------------------------+
+3 rows in set (0.020 ms)
+

Columns

XDB> SELECT * FROM system.columns where database='memory';
++----------+---------+--------+-------+-----+
+| database | table   | column | type  | len |
++----------+---------+--------+-------+-----+
+| memory   | student | id     | INT   | 4   |
+| memory   | student | name   | CHAR  | 16  |
+| memory   | student | age    | INT   | 4   |
+| memory   | student | class  | CHAR  | 16  |
+| memory   | student | score  | FLOAT | 4   |
+| memory   | student | info   | CHAR  | 256 |
+| memory   | teacher | id     | INT   | 4   |
+| memory   | teacher | name   | CHAR  | 16  |
+| memory   | teacher | age    | INT   | 4   |
+| memory   | teacher | info   | CHAR  | 256 |
+| memory   | book    | id     | INT   | 4   |
+| memory   | book    | name   | CHAR  | 64  |
+| memory   | book    | author | CHAR  | 32  |
+| memory   | book    | count  | INT   | 4   |
++----------+---------+--------+-------+-----+
+14 rows in set (0.017 ms)
+

Indexes

XDB> SELECT * FROM system.indexes where database='memory';
++----------+---------+---------+------+----------+
+| database | table   | idx_key | type | col_list |
++----------+---------+---------+------+----------+
+| memory   | student | PRIMARY | HASH | id       |
+| memory   | student | name_2  | HASH | name     |
+| memory   | teacher | PRIMARY | HASH | id       |
+| memory   | teacher | name_2  | HASH | name     |
+| memory   | book    | PRIMARY | HASH | id       |
+| memory   | book    | name_2  | HASH | name     |
++----------+---------+---------+------+----------+
+6 rows in set (0.016 ms)
+

Comments

\ No newline at end of file