【StreamingLLM】トークン数400万、小説40冊を読み込める無限LLM

!pip install torch torchvision torchaudio
!pip install transformers==4.33.0 accelerate datasets evaluate wandb scikit-learn scipy sentencepiece

!git clone https://github.com/mit-han-lab/streaming-llm.git
%cd streaming-llm

!python setup.py develop

%cd examples/
!python run_streaming_llama.py --enable_streaming

Loading model from lmsys/vicuna-13b-v1.3 ...
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama.LlamaTokenizer'>. If you see this, DO NOT PANIC! This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thouroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565
Downloading shards:   0%|                                                        | 0/3 [00:00<?, ?it/s]pytpytorch_model-00001-of-00003.bin: 100%|███████████████████████████| 9.95G/9.95G [45:39<00:00, 3.63MB/s]
pytorch_model-00002-of-00003.bin: 100%|██████████████████████████████| 9.90G/9.90G [40:07<00:00, 4.11MB/s]pytorch_model-00003-of-00003.bin: 100%|██████████████████████████████| 6.18G/6.18G [20:08<00:00, 5.11MB/s]Downloading shards: 100%|████████████████████████████████████████████| 3/3 [1:46:01<00:00, 2120.33s/it]/s]Loading checkpoint shards: 100%|████████████████████████████████████████████| 3/3 [00:19<00:00,  6.49s/it](…)v1.3/resolve/main/generation_config.json: 100%|███████████████████████| 132/132 [00:00<00:00, 12.3kB/s]Loading data from data/mt_bench.jsonl ...
StartRecentKVCache: 4, 2000

USER: Compose an engaging travel blog post about a recent trip to Hawaii, highlighting cultural experiences and must-see attractions.

ASSISTANT: 🌴🌺 Aloha from the beautiful islands of Hawaii! I recently had the opportunity to explore the diverse culture and stunning landscapes of this tropical paradise, and I can't wait to share my adventures with you.

🏝️ First things first, let's talk about the beaches. Hawaii is known for its pristine white sand and crystal clear waters, and I was lucky enough to visit some of the most breathtaking beaches on the islands. My favorite was Waimea Bay on the North Shore of Oahu. The waves were perfect for surfing, and the surrounding cliffs and lush greenery made for a postcard-worthy backdrop.

🌊 But Hawaii isn't just about the beaches. The culture of the islands is rich and vibrant, and I made it a point to immerse myself in it during my trip. I attended a traditional Hawaiian luau on the Big Island, where I learned about the history and customs of the native people. I also tried my hand at hula dancing and even got to sample some delicious Hawaiian cuisine, including poke bowls and shaved ice.

🏯 Another must-see attraction in Hawaii is the Hawaii Volcanoes National Park on the Big Island. The park is home to two active volcanoes, Kilauea and Mauna Loa, and offers a unique opportunity to see the power of nature up close. I hiked through the park and was amazed by the steam vents, lava tubes, and volcanic craters. It was truly a once-in-a-lifetime experience.

🌺 In addition to the natural wonders of Hawaii, the islands are also home to some of the most beautiful gardens and parks in the world. The Maui Tropical Plantation offered a glimpse into the island's agricultural history, while the Hilo Tropical Botanical Garden showcased the stunning flora and fauna of the area.

🏯 In conclusion, Hawaii is a destination that offers something for everyone. From the stunning beaches to the rich culture and unique attractions, there's no shortage of things to see and do on these beautiful islands. If you're planning a trip to Hawaii, be sure to add these must-see attractions to your itinerary. Mahalo for reading, and aloha!

USER: Rewrite your previous response. Start every sentence with the letter A.

ASSISTANT: 🌴🌺 Aloha from the beautiful islands of Hawaii! I recently had the opportunity to explore the diverse culture and stunning landscapes of this tropical paradise, and I can't wait to share my adventures with you.

🏝️ First things first, let's talk about the beaches. Hawaii is known for its pristine white sand and crystal clear waters, and I was lucky enough to visit some of the most breathtaking beaches on the islands. My favorite was Waimea Bay on the North Shore of Oahu. The waves were perfect for surfing, and the surrounding cliffs and lush greenery made for a postcard-worthy backdrop.

🌊 But Hawaii isn't just about the beaches. The culture of the islands is rich and vibrant, and I made it a point to immerse myself in it during my trip. I attended a traditional Hawaiian luau on the Big Island, where I learned about the history and customs of the native people. I also tried my hand at hula dancing and even got to sample some delicious Hawaiian cuisine, including poke bowls and shaved ice.

🌺 In addition to the natural wonders of Hawaii, the islands are also home to some of the most stunning gardens and parks in the world. The Maui Tropical Plantation offered a glimpse into the island's agricultural history, while the Hilo Tropical Botanical Garden showcased the stunning flora and fauna of the area.

🏯 In conclusion, Hawaii is a destination that offers something for everyone. From the stunning beaches to the rich culture and unique attractions, there's no shortage of things to see and do on these beautiful islands. If you're planning a trip to Hawaii, be sure to add these must-see attractions to your itinerary. Mahalo for reading, and aloha!

USER: Draft a professional email seeking your supervisor's feedback on the 'Quarterly Financial Report' you prepared. Ask specifically about the data analysis, presentation style, and the clarity of conclusions drawn. Keep the email short and to the point.

ASSISTANT: 💼 Dear [Supervisor's Name],

I hope this email finds you well. I am writing to seek your feedback on the Quarterly Financial Report that I Traceback (most recent call last):

(省略)

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 136.00 MiB. GPU 0 has a total capacty of 15.99 GiB of which 0 bytes is free. Including non-PyTorch memory, this process has 17179869184.00 GiB memory in use. Of the allocated memory 14.50 GiB is allocated by PyTorch, and 536.37 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF