iT邦幫忙

2022 iThome 鐵人賽

DAY 24
0

今天我們來補充自然語言處理中的一個很重要的概念:Named Entity Recognition(NER)。 一般翻譯為命名實體辨識、命名實體識別,或也有人翻成專有名詞辨識。在自然語言處理中,產品、地點和人等等指涉現實世界的對象稱為命名實體,從文本中提取它們,這個動作就稱為命名實體識別。

我們來看看 Hugging Face 如何簡單快速的來做 NER 吧!

  1. 我們先來準備一段文本,我們一樣用前幾天根據比特幣創世區塊所產生的文本。
sample_text = """
Alistair Darling has been forced to consider a second bailout for banks as the lending drought worsens. 

The Cancellor will decide tithin weeks whether to pump billions more into the economy as evidence mounts that the 37 billion part-nationalisation last yearr has failed to keep credit flowing,

Mr Darling, the former Liberal Democrat chancellor, admitted that the situation had become critical but insisted that there was still time to turn things around. 

He told the BBC that the crisis in the banking sector was the most serious problem facing the economy but also highlighted other issues, such as the falling value of sterling and the threat of inflation. 

"The worst fears about the banking crisis seem not to be panning out," he said, adding that there had not been a single banker arrested or charged over the crash. 

"The economy, the economy"

Mr Darling said "there's been a very, very strong recovery" since the autumn of 2008.

"There are very big problems ahead of us, not least of which is inflation. It is likely to be a very high inflation rate. "

The economy is expected to grow by 0.3% in the quarter to the end of this year.
"""
  1. 接著使用 Pipeline 的方法,把 NER 的功能呼叫進來。
from transformers import pipeline
import pandas as pd


ner = pipeline("ner")
outputs = ner(sample_text)
pd.DataFrame(outputs)

會得到下面圖片中的結果:

koko hugging face azure machine learning

  1. 我們可以看到人名都有被辨識為 PER(person),組織都有被辨識為 ORG (organization),而 MISC 是指 Miscellaneous entities。而前面的 I 是指 inside 的意思,還有另一個常見的字首是 B,是指 Beginning 的意思。
  2. 上圖中的 Democrat 應該是指民主黨,應該要被辨識為 organization 才對,但是這個字也有支持民主的人的意思,因此被歸在 MISC 似乎也不完全是錯誤。

以上就是補充 NLP 中很重要的 NER 的概念,明天我們來做 Q&A 吧!


上一篇
# Day23- Fine-tuned 摘要任務的 transformer
下一篇
# Day25- Hugging Face 問答任務
系列文
變形金剛與抱臉怪---NLP 應用開發之實戰30
圖片
  直播研討會
圖片
{{ item.channelVendor }} {{ item.webinarstarted }} |
{{ formatDate(item.duration) }}
直播中

尚未有邦友留言

立即登入留言