Microsoft’s LongRoPE Breaks the Limit of Context Window of LLMs, Extents it to 2 Million Tokens | Synced
In a new paper LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens, a Microsoft research team introduces LongRoPE, a pioneering method that extends the context window of pre-trained LLMs...
Source: Synced | AI Technology & Industry Review
In a new paper LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens, a Microsoft research team introduces LongRoPE, a pioneering method that extends the context window of pre-trained LLMs to an impressive 2048k tokens while preserving performance at the original short context window.