Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[test stuff][RFC][AIC-py] hf summarization model parser
Very similar to text gen Test: ``` ... { "name": "test_hf_sum", "input": "HMS Duncan was a D-class destroyer ...", # [contents of https://en.wikipedia.org/wiki/HMS_Duncan_(D99)] "metadata": { "model": { "name": "stevhliu/my_awesome_billsum_model", "settings": { "min_length": 100, "max_length": 200, "num_beams": 1 } } } }, ... } ``` ``` import asyncio from aiconfig_extension_hugging_face.local_inference.text_summarization import HuggingFaceTextSummarizationTransformer from aiconfig import AIConfigRuntime, InferenceOptions, CallbackManager # Load the aiconfig. mp = HuggingFaceTextSummarizationTransformer() AIConfigRuntime.register_model_parser(mp, "stevhliu/my_awesome_billsum_model") config = AIConfigRuntime.load("/Users/jonathan/Projects/aiconfig/cookbooks/Getting-Started/travel.aiconfig.json") config.callback_manager = CallbackManager([]) def print_stream(data, _accumulated_data, _index): print(data, end="", flush=True) async def run(): print("Stream") options = InferenceOptions(stream=True, stream_callback=print_stream) out = await config.run("test_hf_sum", options=options) print("Output:\n", out) print("no stream") options = InferenceOptions(stream=False) out = await config.run("test_hf_sum", options=options) print("Output:\n", out) asyncio.run(run()) # OUT: Stream Token indices sequence length is longer than the specified maximum sequence length for this model (2778 > 512). Running this sequence through the model will result in indexing errors /opt/homebrew/Caskroom/miniconda/base/envs/aiconfig/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:430: UserWarning: `num_beams` is set to 1. However, `early_stopping` is set to `True` -- this flag is only used in beam-based generation modes. You should set `num_beams>1` or unset `early_stopping`. warnings.warn( /opt/homebrew/Caskroom/miniconda/base/envs/aiconfig/lib/python3.10/site-packages/transformers/generation/configuration_utils.py:449: UserWarning: `num_beams` is set to 1. However, `length_penalty` is set to `2.0` -- this flag is only used in beam-based generation modes. You should set `num_beams>1` or unset `length_penalty`. warnings.warn( <pad> escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the </s>Output: [ExecuteResult(output_type='execute_result', execution_count=0, data="<pad> escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the </s>", mime_type=None, metadata={})] no stream Output: [ExecuteResult(output_type='execute_result', execution_count=0, data="escorted the 13th Destroyer Flotilla in the Mediterranean and escorited the carrier Argus to Malta during the war . The ship was recalled home to be converted into an escoter destroyer in late 1942. The vessel was repaired and given a refit at Gibraltar on 16 November, and was sold for scrap later that year. The crew of the ship escores the ship to the Middle East, and the ship is a 'disaster' of the .", mime_type=None, metadata={})] ```
- Loading branch information