コンテンツカテゴリ
用語
AIによる要約
各トークンの重要性を計算し、文脈内での関連性を判断するためのメカニズム
コンテンツ
タグ
基本用語
レベル
ツール
ChatGPT
作成日時
Jan 29, 2024 7:32 AM
最終更新日時
Feb 7, 2024 5:23 AM
Attention (machine learning)
Machine learning-based attention is a mechanism mimicking cognitive attention. It calculates "soft" weights for each word, more precisely for its embedding, in the context window. It can do it either in parallel or sequentially. "Soft" weights can change during each runtime, in contrast to "hard" weights, which are (pre-)trained and fine-tuned and remain frozen afterwards.
en.wikipedia.org
