forked from takuyaa/kuromoji.js
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
48 changed files
with
2,341 additions
and
20,634 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
name: Test | ||
|
||
on: | ||
push: | ||
pull_request: | ||
workflow_dispatch: | ||
|
||
jobs: | ||
test: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
|
||
- name: Setup Node | ||
uses: actions/setup-node@v4 | ||
with: | ||
node-version-file: ".node-version" | ||
cache: "npm" | ||
|
||
- shell: bash | ||
run: npm ci | ||
|
||
- run: npm run lint | ||
- run: npm run test |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1 @@ | ||
8.10.0 | ||
22.11.0 |
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,104 +1,53 @@ | ||
kuromoji.js | ||
=========== | ||
# VOICEVOX kuromoji.js | ||
|
||
[![Build Status](https://travis-ci.org/takuyaa/kuromoji.js.svg?branch=master)](https://travis-ci.org/takuyaa/kuromoji.js) | ||
[![Coverage Status](https://coveralls.io/repos/github/takuyaa/kuromoji.js/badge.svg?branch=master)](https://coveralls.io/github/takuyaa/kuromoji.js?branch=master) | ||
[![npm version](https://badge.fury.io/js/kuromoji.svg)](https://badge.fury.io/js/kuromoji) | ||
[![dependencies](https://david-dm.org/takuyaa/kuromoji.js.svg)](https://david-dm.org/takuyaa/kuromoji.js) | ||
[![Code Climate](https://codeclimate.com/github/takuyaa/kuromoji.js/badges/gpa.svg)](https://codeclimate.com/github/takuyaa/kuromoji.js) | ||
[![Downloads](https://img.shields.io/npm/dm/kuromoji.svg)](https://www.npmjs.com/package/kuromoji) | ||
[VOICEVOXエディタ](https://github.com/VOICEVOX/voicevox)のモック用にチューニングした[kuromoji.js](https://github.com/takuyaa/kuromoji.js)です。 | ||
ESModuleとして利用できます。 | ||
|
||
JavaScript implementation of Japanese morphological analyzer. | ||
This is a pure JavaScript porting of [Kuromoji](https://www.atilika.com/ja/kuromoji/). | ||
機能追加はしない予定で、VOICEVOXエディタ内で不都合が生じない限り積極的にメンテナンスもしない予定です。 | ||
標準的なライブラリやフレームワークを用いた開発環境の改善は歓迎します。 | ||
|
||
You can see how kuromoji.js works in [demo site](https://takuyaa.github.io/kuromoji.js/demo/tokenize.html). | ||
## 使い方 | ||
|
||
```sh | ||
npm install github:VOICEVOX/kuromoji.js | ||
``` | ||
|
||
Directory | ||
--------- | ||
```ts | ||
import { builder, IpadicFeatures, Tokenizer } from "kuromoji"; | ||
|
||
Directory tree is as follows: | ||
let _tokenizer: Tokenizer<IpadicFeatures>; | ||
|
||
build/ | ||
kuromoji.js -- JavaScript file for browser (Browserified) | ||
demo/ -- Demo | ||
dict/ -- Dictionaries for tokenizer (gzipped) | ||
example/ -- Examples to use in Node.js | ||
src/ -- JavaScript source | ||
test/ -- Unit test | ||
// ブラウザで使う場合 | ||
builder({ | ||
nodeOrBrowser: "browser", | ||
dicPath: "https://url/to/dict", | ||
}).build((err: Error, tokenizer: Tokenizer<IpadicFeatures>) => { | ||
_tokenizer = tokenizer; | ||
}); | ||
|
||
// Node.jsで使う場合 | ||
builder({ | ||
nodeOrBrowser: "node", | ||
dicPath: "node_modules/kuromoji/dict", | ||
}).build((err: Error, tokenizer: Tokenizer<IpadicFeatures>) => { | ||
_tokenizer = tokenizer; | ||
}); | ||
``` | ||
|
||
Usage | ||
----- | ||
## 開発環境 | ||
|
||
You can tokenize sentences with only 5 lines of code. | ||
If you need working examples, you can see the files under the demo or example directory. | ||
```sh | ||
npm i | ||
``` | ||
|
||
## テスト | ||
|
||
### Node.js | ||
```sh | ||
npm run test | ||
``` | ||
|
||
Install with npm package manager: | ||
## フォーマット | ||
|
||
npm install kuromoji | ||
|
||
Load this library as follows: | ||
|
||
var kuromoji = require("kuromoji"); | ||
|
||
You can prepare tokenizer like this: | ||
|
||
kuromoji.builder({ dicPath: "path/to/dictionary/dir/" }).build(function (err, tokenizer) { | ||
// tokenizer is ready | ||
var path = tokenizer.tokenize("すもももももももものうち"); | ||
console.log(path); | ||
}); | ||
|
||
|
||
|
||
### Browser | ||
|
||
You only need the build/kuromoji.js and dict/*.dat.gz files | ||
|
||
Install with Bower package manager: | ||
|
||
bower install kuromoji | ||
|
||
Or you can use the kuromoji.js file and dictionary files from the GitHub repository. | ||
|
||
In your HTML: | ||
|
||
<script src="url/to/kuromoji.js"></script> | ||
|
||
In your JavaScript: | ||
|
||
kuromoji.builder({ dicPath: "/url/to/dictionary/dir/" }).build(function (err, tokenizer) { | ||
// tokenizer is ready | ||
var path = tokenizer.tokenize("すもももももももものうち"); | ||
console.log(path); | ||
}); | ||
|
||
|
||
API | ||
--- | ||
|
||
The function tokenize() returns an JSON array like this: | ||
|
||
[ { | ||
word_id: 509800, // 辞書内での単語ID | ||
word_type: 'KNOWN', // 単語タイプ(辞書に登録されている単語ならKNOWN, 未知語ならUNKNOWN) | ||
word_position: 1, // 単語の開始位置 | ||
surface_form: '黒文字', // 表層形 | ||
pos: '名詞', // 品詞 | ||
pos_detail_1: '一般', // 品詞細分類1 | ||
pos_detail_2: '*', // 品詞細分類2 | ||
pos_detail_3: '*', // 品詞細分類3 | ||
conjugated_type: '*', // 活用型 | ||
conjugated_form: '*', // 活用形 | ||
basic_form: '黒文字', // 基本形 | ||
reading: 'クロモジ', // 読み | ||
pronunciation: 'クロモジ' // 発音 | ||
} ] | ||
|
||
(This is defined in src/util/IpadicFormatter.js) | ||
|
||
See also [JSDoc page](https://takuyaa.github.io/kuromoji.js/jsdoc/) in details. | ||
```sh | ||
npm run fmt | ||
``` |
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.