Uses spaCy to extract part-of-speech (POS) tags from tokenized text. Returns a data frame with token-level POS annotations.
Usage
extract_pos_tags(
tokens,
include_lemma = TRUE,
include_entity = FALSE,
include_dependency = FALSE,
model = "en_core_web_sm"
)Arguments
- tokens
A quanteda tokens object or character vector of texts.
- include_lemma
Logical; include lemmatized forms (default: TRUE).
- include_entity
Logical; include named entity recognition (default: FALSE).
- include_dependency
Logical; include dependency parsing (default: FALSE).
- model
Character; spaCy model to use (default: "en_core_web_sm").
Value
A data frame with columns:
doc_id: Document identifiersentence_id: Sentence number within documenttoken_id: Token position within sentencetoken: Original tokenpos: Universal POS tag (e.g., NOUN, VERB, ADJ)tag: Detailed POS tag (e.g., NN, VBD, JJ)lemma: Lemmatized form (if include_lemma = TRUE)entity: Named entity type (if include_entity = TRUE)head_token_id: Head token in dependency tree (if include_dependency = TRUE)dep_rel: Dependency relation type, e.g., nsubj, dobj (if include_dependency = TRUE)
Details
This function requires the Python with spaCy installed. If spaCy is not initialized, this function will attempt to initialize it with the specified model.
Examples
if (interactive()) {
tokens <- quanteda::tokens(TextAnalysisR::SpecialEduTech$abstract[1])
pos_data <- extract_pos_tags(tokens)
print(pos_data)
}
#> doc_id sentence_id token_id token pos tag lemma
#> 1 text1 1 1 Notes VERB VBZ note
#> 2 text1 1 2 that SCONJ IN that
#> 3 text1 1 3 the DET DT the
#> 4 text1 1 4 ALP PROPN NNP ALP
#> 5 text1 1 5 minicalculator NOUN NN minicalculator
#> 6 text1 1 6 program NOUN NN program
#> 7 text1 1 7 for ADP IN for
#> 8 text1 1 8 elementary ADJ JJ elementary
#> 9 text1 1 9 mathematics NOUN NNS mathematic
#> 10 text1 1 10 has AUX VBZ have
#> 11 text1 1 11 worked VERB VBN work
#> 12 text1 1 12 well ADV RB well
#> 13 text1 1 13 in ADP IN in
#> 14 text1 1 14 a DET DT a
#> 15 text1 1 15 clinical ADJ JJ clinical
#> 16 text1 1 16 setting NOUN NN setting
#> 17 text1 1 17 with ADP IN with
#> 18 text1 1 18 learning NOUN NN learning
#> 19 text1 1 19 - PUNCT HYPH -
#> 20 text1 1 20 disabled ADJ JJ disabled
#> 21 text1 1 21 youngsters NOUN NNS youngster
#> 22 text1 1 22 with ADP IN with
#> 23 text1 1 23 perceptual ADJ JJ perceptual
#> 24 text1 1 24 and CCONJ CC and
#> 25 text1 1 25 / SYM SYM /
#> 26 text1 1 26 or CCONJ CC or
#> 27 text1 1 27 memory NOUN NN memory
#> 28 text1 1 28 deficits NOUN NNS deficit
#> 29 text1 1 29 who PRON WP who
#> 30 text1 1 30 can AUX MD can
#> 31 text1 1 31 , PUNCT , ,
#> 32 text1 1 32 nonetheless ADV RB nonetheless
#> 33 text1 1 33 , PUNCT , ,
#> 34 text1 1 34 demonstrate VERB VB demonstrate
#> 35 text1 1 35 an DET DT an
#> 36 text1 1 36 understanding NOUN NN understanding
#> 37 text1 1 37 of ADP IN of
#> 38 text1 1 38 basic ADJ JJ basic
#> 39 text1 1 39 math NOUN NN math
#> 40 text1 1 40 concepts NOUN NNS concept
#> 41 text1 1 41 . PUNCT . .
#> 42 text1 2 1 Although SCONJ IN although
#> 43 text1 2 2 the DET DT the
#> 44 text1 2 3 approach NOUN NN approach
#> 45 text1 2 4 appears VERB VBZ appear
#> 46 text1 2 5 to PART TO to
#> 47 text1 2 6 have AUX VB have
#> 48 text1 2 7 tremendous ADJ JJ tremendous
#> 49 text1 2 8 potential NOUN NN potential
#> 50 text1 2 9 for ADP IN for
#> 51 text1 2 10 learning NOUN NN learning
#> 52 text1 2 11 - PUNCT HYPH -
#> 53 text1 2 12 disabled ADJ JJ disabled
#> 54 text1 2 13 math NOUN NN math
#> 55 text1 2 14 students NOUN NNS student
#> 56 text1 2 15 throughout ADP IN throughout
#> 57 text1 2 16 all DET DT all
#> 58 text1 2 17 realms NOUN NNS realm
#> 59 text1 2 18 of ADP IN of
#> 60 text1 2 19 education NOUN NN education
#> 61 text1 2 20 , PUNCT , ,
#> 62 text1 2 21 caution NOUN NN caution
#> 63 text1 2 22 is AUX VBZ be
#> 64 text1 2 23 advised VERB VBN advise
#> 65 text1 2 24 when SCONJ WRB when
#> 66 text1 2 25 choosing VERB VBG choose
#> 67 text1 2 26 the DET DT the
#> 68 text1 2 27 time NOUN NN time
#> 69 text1 2 28 and CCONJ CC and
#> 70 text1 2 29 circumstances NOUN NNS circumstance
#> 71 text1 2 30 of ADP IN of
#> 72 text1 2 31 its PRON PRP$ its
#> 73 text1 2 32 instigation NOUN NN instigation
#> 74 text1 2 33 . PUNCT . .
#> 75 text1 3 1 Also ADV RB also
#> 76 text1 3 2 presented VERB VBN present
#> 77 text1 3 3 are AUX VBP be
#> 78 text1 3 4 the DET DT the
#> 79 text1 3 5 advantages NOUN NNS advantage
#> 80 text1 3 6 found VERB VBN find
#> 81 text1 3 7 when SCONJ WRB when
#> 82 text1 3 8 the DET DT the
#> 83 text1 3 9 ALP PROPN NNP ALP
#> 84 text1 3 10 approach NOUN NN approach
#> 85 text1 3 11 was AUX VBD be
#> 86 text1 3 12 compared VERB VBN compare
#> 87 text1 3 13 to ADP IN to
#> 88 text1 3 14 the DET DT the
#> 89 text1 3 15 traditional ADJ JJ traditional
#> 90 text1 3 16 approach NOUN NN approach
#> 91 text1 3 17 of ADP IN of
#> 92 text1 3 18 subtracting VERB VBG subtract
#> 93 text1 3 19 fractions NOUN NNS fraction
#> 94 text1 3 20 . PUNCT . .