Skip to content

Commit

Permalink
Merge branch 'main' into linter-fix
Browse files Browse the repository at this point in the history
  • Loading branch information
chmjkb authored Nov 8, 2024
2 parents a7aac4c + a84c1eb commit 0d333d6
Show file tree
Hide file tree
Showing 7 changed files with 204 additions and 55 deletions.
16 changes: 11 additions & 5 deletions android/src/main/java/com/rnexecutorch/Fetcher.kt
Original file line number Diff line number Diff line change
Expand Up @@ -106,13 +106,14 @@ class Fetcher {
/*
Fetching model and tokenizer file
1. Extract file name from provided URL
2. Check if the file has a valid extension
2. If file name contains / it means that the file is local and we should return the path
3. Check if the file has a valid extension
a. For tokenizer, the extension should be .bin
b. For model, the extension should be .pte
3. Check if models directory exists, if not create it
4. Check if the file already exists in the models directory, if yes return the path
5. If the file does not exist, and is a tokenizer, fetch the file
6. If the file is a model, fetch the file with ProgressResponseBody
4. Check if models directory exists, if not create it
5. Check if the file already exists in the models directory, if yes return the path
6. If the file does not exist, and is a tokenizer, fetch the file
7. If the file is a model, fetch the file with ProgressResponseBody
*/
val fileName: String

Expand All @@ -123,6 +124,11 @@ class Fetcher {
return
}

if(fileName.contains("/")){
onComplete(fileName, null)
return
}

if (!hasValidExtension(fileName, resourceType)) {
onComplete(null, Exception("invalid_resource_extension"))
return
Expand Down
10 changes: 5 additions & 5 deletions docs/docs/guides/exporting-llama.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@ sidebar_position: 2
In order to make the process of export as simple as possible for you, we created a script that runs a Docker container and exports the model.

## Steps to export Llama
### 1. Create an Account:
### 1. Create an Account
Get a [HuggingFace](https://huggingface.co/) account. This will allow you to download needed files. You can also use the [official Llama website](https://www.llama.com/llama-downloads/).

### 2. Select a Model:
### 2. Select a Model
Pick the model that suits your needs. Before you download it, you'll need to accept a license. For best performance, we recommend using Spin-Quant or QLoRA versions of the model:
- [Llama 3.2 3B](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct/tree/main/original)
- [Llama 3.2 1B](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct/tree/main/original)
Expand All @@ -18,16 +18,16 @@ Pick the model that suits your needs. Before you download it, you'll need to acc
- [Llama 3.2 3B QLoRA](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct-QLORA_INT4_EO8/tree/main)
- [Llama 3.2 1B QLoRA](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct-QLORA_INT4_EO8/tree/main)

### 3. Download Files:
### 3. Download Files
Download the `consolidated.00.pth`, `params.json` and `tokenizer.model` files. If you can't see them, make sure to check the `original` directory.

### 4. Rename the Tokenizer File:
### 4. Rename the Tokenizer File
Rename the `tokenizer.model` file to `tokenizer.bin` as required by the library:
```bash
mv tokenizer.model tokenizer.bin
```

### 5. Run the Export Script:
### 5. Run the Export Script
Navigate to the `llama_export` directory and run the following command:
```bash
./build_llama_binary.sh --model-path /path/to/consolidated.00.pth --params-path /path/to/params.json
Expand Down
47 changes: 2 additions & 45 deletions docs/docusaurus.config.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,3 @@
/*
In swizzled components look for "SWM -" string to see our modifications
*/

const lightCodeTheme = require('./src/theme/CodeBlock/highlighting-light.js');
const darkCodeTheme = require('./src/theme/CodeBlock/highlighting-dark.js');
// @ts-check
Expand Down Expand Up @@ -37,8 +33,8 @@ const config = {
theme: {
customCss: require.resolve('./src/css/index.css'),
},
googleAnalytics: {
trackingID: '...',
gtag: {
trackingID: 'G-TJND8QJM9P',
anonymizeIP: true,
},
}),
Expand Down Expand Up @@ -87,45 +83,6 @@ const config = {
indexName: 'react-native-executorch',
},
}),
plugins: [
// ...[
// process.env.NODE_ENV === 'production' && '@docusaurus/plugin-debug',
// ].filter(Boolean),
// async function reanimatedDocusaurusPlugin(context, options) {
// return {
// name: 'react-native-reanimated/docusaurus-plugin',
// configureWebpack(config, isServer, utils) {
// const processMock = !isServer ? { process: { env: {} } } : {};
// const raf = require('raf');
// raf.polyfill();
// return {
// mergeStrategy: {
// 'resolve.extensions': 'prepend',
// },
// plugins: [
// new webpack.DefinePlugin({
// ...processMock,
// __DEV__: 'false',
// setImmediate: () => {},
// }),
// ],
// module: {
// rules: [
// {
// test: /\.txt/,
// type: 'asset/source',
// },
// ],
// },
// resolve: {
// alias: { 'react-native$': 'react-native-web' },
// extensions: ['.web.js', '...'],
// },
// };
// },
// };
// },
],
};

module.exports = config;
22 changes: 22 additions & 0 deletions docs/src/pages/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import styles from './styles.module.css';
import Layout from '@theme/Layout';
import HomepageStartScreen from '@site/src/components/Hero/StartScreen';
import { HireUsSection } from '@swmansion/t-rex-ui';
import LogoHero from '@site/static/img/logo-hero.svg';
import HomepageButton from '@site/src/components/HomepageButton';

function Home() {
return (
Expand All @@ -13,6 +15,26 @@ function Home() {
<div className={styles.container}>
<HomepageStartScreen />
</div>
<div
style={{
marginTop: '300px',
marginBottom: '350px',
gap: '30px',
display: 'flex',
flexDirection: 'column',
justifyContent: 'center',
alignItems: 'center',
}}
>
<LogoHero />
<h2>This site is under development</h2>
<a href="/react-native-executorch/docs/fundamentals/getting-started">
<HomepageButton
href="/react-native-executorch/docs/fundamentals/getting-started"
title="Checkout the docs"
/>
</a>
</div>
<div className={styles.container}>
<HireUsSection
href={
Expand Down
34 changes: 34 additions & 0 deletions docs/static/img/logo-hero.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 0d333d6

Please sign in to comment.