Uses spaCy to extract named entities (NER) from tokenized text. Returns a data frame with token-level entity annotations.
Usage
extract_named_entities(
tokens,
include_pos = TRUE,
include_lemma = TRUE,
model = "en_core_web_sm"
)Value
A data frame with columns:
doc_id: Document identifiertoken: Original tokenentity: Named entity type (e.g., PERSON, ORG, GPE)pos: Universal POS tag (if include_pos = TRUE)lemma: Lemmatized form (if include_lemma = TRUE)
Details
This function requires the Python with spaCy installed. If spaCy is not initialized, this function will attempt to initialize it with the specified model.
Examples
if (interactive()) {
tokens <- quanteda::tokens(TextAnalysisR::SpecialEduTech$abstract[1])
entity_data <- extract_named_entities(tokens)
print(entity_data)
}
#> doc_id sentence_id token_id token pos tag lemma
#> 1 text1 1 1 Notes VERB VBZ note
#> 2 text1 1 2 that SCONJ IN that
#> 3 text1 1 3 the DET DT the
#> 4 text1 1 4 ALP PROPN NNP ALP
#> 5 text1 1 5 minicalculator NOUN NN minicalculator
#> 6 text1 1 6 program NOUN NN program
#> 7 text1 1 7 for ADP IN for
#> 8 text1 1 8 elementary ADJ JJ elementary
#> 9 text1 1 9 mathematics NOUN NNS mathematic
#> 10 text1 1 10 has AUX VBZ have
#> 11 text1 1 11 worked VERB VBN work
#> 12 text1 1 12 well ADV RB well
#> 13 text1 1 13 in ADP IN in
#> 14 text1 1 14 a DET DT a
#> 15 text1 1 15 clinical ADJ JJ clinical
#> 16 text1 1 16 setting NOUN NN setting
#> 17 text1 1 17 with ADP IN with
#> 18 text1 1 18 learning NOUN NN learning
#> 19 text1 1 19 - PUNCT HYPH -
#> 20 text1 1 20 disabled ADJ JJ disabled
#> 21 text1 1 21 youngsters NOUN NNS youngster
#> 22 text1 1 22 with ADP IN with
#> 23 text1 1 23 perceptual ADJ JJ perceptual
#> 24 text1 1 24 and CCONJ CC and
#> 25 text1 1 25 / SYM SYM /
#> 26 text1 1 26 or CCONJ CC or
#> 27 text1 1 27 memory NOUN NN memory
#> 28 text1 1 28 deficits NOUN NNS deficit
#> 29 text1 1 29 who PRON WP who
#> 30 text1 1 30 can AUX MD can
#> 31 text1 1 31 , PUNCT , ,
#> 32 text1 1 32 nonetheless ADV RB nonetheless
#> 33 text1 1 33 , PUNCT , ,
#> 34 text1 1 34 demonstrate VERB VB demonstrate
#> 35 text1 1 35 an DET DT an
#> 36 text1 1 36 understanding NOUN NN understanding
#> 37 text1 1 37 of ADP IN of
#> 38 text1 1 38 basic ADJ JJ basic
#> 39 text1 1 39 math NOUN NN math
#> 40 text1 1 40 concepts NOUN NNS concept
#> 41 text1 1 41 . PUNCT . .
#> 42 text1 2 1 Although SCONJ IN although
#> 43 text1 2 2 the DET DT the
#> 44 text1 2 3 approach NOUN NN approach
#> 45 text1 2 4 appears VERB VBZ appear
#> 46 text1 2 5 to PART TO to
#> 47 text1 2 6 have AUX VB have
#> 48 text1 2 7 tremendous ADJ JJ tremendous
#> 49 text1 2 8 potential NOUN NN potential
#> 50 text1 2 9 for ADP IN for
#> 51 text1 2 10 learning NOUN NN learning
#> 52 text1 2 11 - PUNCT HYPH -
#> 53 text1 2 12 disabled ADJ JJ disabled
#> 54 text1 2 13 math NOUN NN math
#> 55 text1 2 14 students NOUN NNS student
#> 56 text1 2 15 throughout ADP IN throughout
#> 57 text1 2 16 all DET DT all
#> 58 text1 2 17 realms NOUN NNS realm
#> 59 text1 2 18 of ADP IN of
#> 60 text1 2 19 education NOUN NN education
#> 61 text1 2 20 , PUNCT , ,
#> 62 text1 2 21 caution NOUN NN caution
#> 63 text1 2 22 is AUX VBZ be
#> 64 text1 2 23 advised VERB VBN advise
#> 65 text1 2 24 when SCONJ WRB when
#> 66 text1 2 25 choosing VERB VBG choose
#> 67 text1 2 26 the DET DT the
#> 68 text1 2 27 time NOUN NN time
#> 69 text1 2 28 and CCONJ CC and
#> 70 text1 2 29 circumstances NOUN NNS circumstance
#> 71 text1 2 30 of ADP IN of
#> 72 text1 2 31 its PRON PRP$ its
#> 73 text1 2 32 instigation NOUN NN instigation
#> 74 text1 2 33 . PUNCT . .
#> 75 text1 3 1 Also ADV RB also
#> 76 text1 3 2 presented VERB VBN present
#> 77 text1 3 3 are AUX VBP be
#> 78 text1 3 4 the DET DT the
#> 79 text1 3 5 advantages NOUN NNS advantage
#> 80 text1 3 6 found VERB VBN find
#> 81 text1 3 7 when SCONJ WRB when
#> 82 text1 3 8 the DET DT the
#> 83 text1 3 9 ALP PROPN NNP ALP
#> 84 text1 3 10 approach NOUN NN approach
#> 85 text1 3 11 was AUX VBD be
#> 86 text1 3 12 compared VERB VBN compare
#> 87 text1 3 13 to ADP IN to
#> 88 text1 3 14 the DET DT the
#> 89 text1 3 15 traditional ADJ JJ traditional
#> 90 text1 3 16 approach NOUN NN approach
#> 91 text1 3 17 of ADP IN of
#> 92 text1 3 18 subtracting VERB VBG subtract
#> 93 text1 3 19 fractions NOUN NNS fraction
#> 94 text1 3 20 . PUNCT . .
#> entity
#> 1
#> 2
#> 3
#> 4 PROGRAM_B
#> 5
#> 6
#> 7
#> 8
#> 9
#> 10
#> 11
#> 12
#> 13
#> 14
#> 15
#> 16
#> 17
#> 18
#> 19
#> 20
#> 21
#> 22
#> 23
#> 24
#> 25
#> 26
#> 27
#> 28
#> 29
#> 30
#> 31
#> 32
#> 33
#> 34
#> 35
#> 36
#> 37
#> 38
#> 39
#> 40
#> 41
#> 42
#> 43
#> 44
#> 45
#> 46
#> 47
#> 48
#> 49
#> 50
#> 51
#> 52
#> 53
#> 54
#> 55
#> 56
#> 57
#> 58
#> 59
#> 60
#> 61
#> 62
#> 63
#> 64
#> 65
#> 66
#> 67
#> 68
#> 69
#> 70
#> 71
#> 72
#> 73
#> 74
#> 75
#> 76
#> 77
#> 78
#> 79
#> 80
#> 81
#> 82
#> 83 PROGRAM_B
#> 84
#> 85
#> 86
#> 87
#> 88
#> 89
#> 90
#> 91
#> 92
#> 93
#> 94