Calculate Document Similarity with Fallbacks

Calculates document similarity with fallback methods and diagnostics. Attempts embeddings first, falls back to Jaccard similarity if needed.

Usage

calculate_similarity_robust(
  texts,
  method = "embeddings",
  embedding_model = "all-MiniLM-L6-v2",
  cache_embeddings = TRUE,
  min_word_length = 3,
  doc_names = NULL
)

Arguments

texts

Character vector of texts

method

Similarity method ("embeddings" or "jaccard")

embedding_model

Model name for embeddings (default: "all-MiniLM-L6-v2")

cache_embeddings

Logical, cache embeddings (default: TRUE)

min_word_length

Minimum word length for Jaccard (default: 3)

doc_names

Optional document names

Value

List containing similarity matrix, method used, embeddings, and diagnostics

Examples

if (interactive()) {
  abstracts <- TextAnalysisR::SpecialEduTech$abstract[1:5]
  similarity_result <- calculate_similarity_robust(abstracts)
  print(similarity_result$similarity_matrix)
  print(similarity_result$diagnostics)
}
#>           doc1      doc2      doc3      doc4      doc5
#> doc1 1.0000000 0.4501718 0.5835429 0.3628276 0.2077256
#> doc2 0.4501718 1.0000000 0.7147079 0.4205060 0.2643311
#> doc3 0.5835429 0.7147079 1.0000000 0.4971649 0.3400415
#> doc4 0.3628276 0.4205060 0.4971649 1.0000000 0.6805842
#> doc5 0.2077256 0.2643311 0.3400415 0.6805842 1.0000000
#> $attempted_methods
#> [1] "embeddings"
#> 
#> $warnings
#> character(0)
#> 
#> $computation_time
#> [1] 2.542172
#>