LLMs: Embeddings to Text

In this short notebook we are exploring the capabilities of a relativly new concept called Text Embeddings Reveal
Data Science
Machine Learning
Python
Generative AI
Natural Language Processing
Large Language Models
Author

Daniel Fat

Published

January 5, 2024

Introduction

In this Jupyter notebook, we explore an innovative concept in the realm of Large Language Models (LLMs) and text embeddings, drawing inspiration from the recent paper “Text Embeddings Reveal (Almost) As Much As Text”. This paper uncovers the surprising capabilities of text embeddings to reveal original text, challenging previous assumptions about their information content.

Background Research

Our journey begins with an examination of the paper’s findings, which highlight the privacy implications and technical feasibilities of reconstructing text from embeddings. As embeddings are widely used in various NLP tasks, understanding their ability to unveil private information becomes crucial.

How Does This Work?

At the core of our exploration is the technique of embedding inversion, a process of transforming dense text embeddings back into the original text. This approach is a form of controlled text generation, aiming to align generated text with specific points in the latent space represented by the embeddings. This novel concept has significant implications for our understanding of embeddings as more than abstract representations.

In the following sections, we will dive deeper into these concepts and experiment with the ‘vec2text’ Python package, a direct application of the research in the paper.

Let’s start

First, installing the python package

# -q for quiet
%pip install vec2text python-dotenv -q

after that, we will import the necessary libraries

import os
import nltk
import torch
import dotenv
import openai
import vec2text
import pandas as pd

dotenv.load_dotenv(dotenv.find_dotenv())
openai.api_key = os.getenv('OPENAI_API_KEY')

Embedding Inversion

Embedding inversion is a process of transforming dense text embeddings back into the original text. This approach is a form of controlled text generation, aiming to align generated text with specific points in the latent space represented by the embeddings.

We will use the vec2text python package to perform embedding inversion. This package is a direct application of the research in the paper.

Let’s create some variables for our experiment

  • device - the device to run the model on [ ‘cpu’, ‘cuda’, ‘mps’ ]
  • embedding_model - the embedding model to use
  • corrector - the corrector model to use for inversion - this model must match the embedding model
# cuda if available, else cpu, or mps for Apple M1
device = 'mps'
embeddings_model = 'text-embedding-ada-002'
corrector = vec2text.load_corrector(embeddings_model)
/Users/cristianexer/anaconda3/envs/vec2text/lib/python3.11/site-packages/transformers/models/t5/tokenization_t5_fast.py:160: FutureWarning: This tokenizer was incorrectly instantiated with a model max length of 512 which will be corrected in Transformers v5.
For now, this behavior is kept to avoid breaking backwards compatibility when padding/encoding with `truncation is True`.
- Be aware that you SHOULD NOT rely on t5-base automatically truncating your input to 512 when padding/encoding.
- If you want to encode/pad to sequences longer than 512 you can either instantiate this tokenizer with `model_max_length` or pass `max_length` when encoding/padding.
- To avoid this warning, please instantiate this tokenizer with `model_max_length` set to your preferred value.
  warnings.warn(

Text Samples

We will use the following text samples for our experiment:

# for the text samples, I decided to copy some of the text from the paper mentioned above

text_samples = [
    'How much private information do text embeddings reveal about the original text?',
    'We investigate the problem of embedding inversion, reconstructing the full text represented in dense text embeddings.',
    'We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space.',
    
    # then I want to add something very specific, like numbers, emails, etc.
    'My phone number is 555-555-5555',
    
    # my email address :)
    'daniel.fat@icloud.com',
    
    # a random address
    '1234 Main Street, Anytown, USA',
    
]

then we will call openai API to get the embeddings for the text samples

response = openai.embeddings.create(
    input=text_samples,
    model=embeddings_model,
    encoding_format="float",  # override default base64 encoding...
)
response
CreateEmbeddingResponse(data=[Embedding(embedding=[-0.0037350017, -0.0067985803, 0.018610902, -0.032903105, 0.00553671, -0.0025541068, -0.014413665, -0.0028122168, -0.022889117, -0.0154528525, 0.0044300426, 0.044347662, -0.004838295, 0.004112888, -0.0001668014, 0.037518717, 0.019474642, 0.024171231, 0.02114814, -0.026128141, -0.02516993, 0.0031394933, -0.045103434, 0.010047728, -0.0038598392, -0.013941308, 0.032336276, -0.017301796, 0.0077129304, -0.040811725, 0.027747655, -0.0058369944, -0.03355091, -3.3370823e-05, -0.027545216, 0.005023864, -0.00052254595, -0.012949356, 0.046372052, 0.004193864, 0.005067726, 0.016478544, 0.007159597, 0.002085123, -0.00555358, -0.00028931923, 0.025088955, -0.013522933, 0.013340738, 0.0081785405, 0.022200823, 0.025277898, -0.013489193, -0.002982603, -0.035251398, 0.0056210593, -0.015169438, 0.02739676, -0.00322553, 0.013846836, 0.03438766, 0.005047482, -0.007625207, 0.0032609568, -0.024022775, -0.0120653715, -0.019839033, 0.01143781, 0.025952695, 0.010020737, 0.009224475, 0.014373178, -0.009858785, -0.027086355, 0.0015064845, -0.004750571, 0.005249921, 0.0025777249, -0.016249113, 0.0010628054, -0.00097423827, 0.002533863, 0.0020446351, 0.022052368, 0.007092117, 0.0030888836, 0.02214684, 0.021741962, -0.009008541, 0.00062587427, 0.012227323, 0.012153095, 0.0059719537, 0.0032592698, -0.014589112, -0.0021003059, -0.017301796, 0.017450253, 0.003431343, -0.02353692, 0.012139599, 0.011275859, -0.03714083, -0.0117212245, -0.020324888, 0.02440066, 0.018840333, -0.01942066, 0.0037316277, 0.014859031, -0.016775455, 0.021134645, 0.0067108567, -0.024360174, -0.007159597, 0.003434717, 0.014103259, -0.0028594525, -0.029124241, -0.028935298, 0.037545707, -0.012544477, 0.024819035, -0.01117464, 0.027855622, 0.016559519, -0.006117035, -0.029772047, -0.0118022, -0.008637402, -0.0051520756, 0.018152041, 0.018179033, 0.0018101431, -0.025210418, 0.037194815, 0.001703019, 0.003417847, -0.009750818, -0.017868627, 0.01785513, 0.017666187, -0.01160651, 0.013421713, -0.023617897, 0.00917724, -0.009858785, 0.023577409, 0.027423752, 0.0028408957, 0.0019417285, -0.0140087865, 0.003427969, 0.025196923, -0.024360174, 0.008421468, -0.004649352, 0.024994483, -0.017625699, -0.013293502, -0.011694233, 0.02603367, 0.009305451, 7.5018455e-05, 0.0058876043, 0.04075774, -0.0071798405, -0.005705409, 0.006123783, -0.014278706, -0.031013673, 0.018678382, -0.03611514, -0.013482445, 0.006208133, -0.0013639336, 0.028530419, -0.01150529, -9.7740136e-05, -0.035764243, -0.018179033, 0.02580424, 0.012584965, 0.0072270767, -0.008259516, -0.0086778905, 0.014427161, -0.003954311, 0.014629601, -0.017963098, -0.0037788635, 0.01612765, 0.0024967492, -0.004848417, -0.64478207, -0.012888623, 0.034900505, -0.023496434, 0.018340984, 0.011194883, 0.003446526, 0.008718378, -0.03870636, 0.047640674, -0.0017578463, -0.002280814, -0.019393668, -0.002088497, 0.009568622, -0.012315046, 0.0042208554, -0.011431063, 0.021161636, 0.0003447791, -0.0076656947, 0.038598392, 0.014373178, 0.012382526, 0.010897973, 0.0060664257, -0.0016532528, -0.04272815, -0.012895372, 0.0070583774, -0.015844235, 0.021526026, 0.03414473, 0.012193583, 0.041972376, -0.012794152, 0.008488947, 0.008488947, -0.016019681, 0.023091555, -0.04272815, -0.037059855, 0.011559274, 0.0051487014, -0.008340492, 0.015439357, 0.0058538644, -0.005337645, 0.0047033355, 0.004352441, -0.013482445, 0.014116755, -0.0042647175, 0.0060326857, 0.021458546, 0.0030348997, 0.04102766, -0.010513338, 0.025048466, -0.0024393913, 0.0054489863, 0.020176431, -0.0041803676, -0.022538222, 0.0023584156, 0.019312691, 0.004163498, 0.024184726, 0.0009986997, -0.035089448, 0.021728465, 0.026384564, -0.0030568307, -0.0042444733, -0.005081222, 0.03544034, 0.053578887, -0.007739922, -0.005759393, 0.0076117106, 0.019123748, -0.008111061, -0.020162936, 0.021647489, 0.0065556536, -0.004791059, -0.015385373, -0.014089762, 0.014616104, -0.005563702, -0.0070651253, 0.040649775, -0.036223106, -0.03239026, 0.005236425, 0.024103751, -0.013070819, -0.01243651, 0.0007422768, -0.0014406918, 0.0058538644, -0.003883457, 0.023280498, -2.923242e-05, 0.02616863, -0.00022078516, -0.012112607, -0.0017561594, 0.014993991, -0.018165536, 0.0017291674, 0.006447686, 0.001843883, -0.029367168, -0.019879522, -0.026357573, 0.037464734, 0.039651074, -0.0051858155, -0.004568376, -0.009082768, 0.009130004, 0.011485046, 0.011761713, 0.020783748, 0.008542931, -0.00090507156, -0.008542931, 0.01633009, 0.01319903, 0.02969107, -0.028665379, 0.018233016, -0.007355288, -0.001823639, 0.008138052, 0.026708467, -0.017409764, -0.021337083, -0.0012854884, -0.028341476, 0.0033486804, 0.014778055, -0.018381473, -0.025763752, -0.028935298, 0.008455208, 0.013995291, 0.008151549, -0.024144238, 0.01842196, -0.014143746, -0.00049091486, -0.0004976628, 0.006464556, -0.002756546, -0.0032474608, -0.035116438, -0.007294556, -0.02543985, -0.0013301937, 0.047100835, -0.004335571, 0.003701262, 0.014454153, -0.015884724, 0.017801147, 0.009426915, -0.0072135804, -0.023132043, 0.010587566, -0.04213433, 0.0051082135, 0.01592521, -0.006903174, 0.020986188, -0.010391874, -0.004899027, 0.0016633747, -0.025547817, 0.010331143, 0.0004344006, -0.0010973887, 0.0011058237, 0.03328099, 0.0099667525, 0.00505423, 0.022538222, -0.003448213, 0.002274066, -0.00057484274, -0.0068188244, -0.013570169, 0.02533188, 0.002977542, -0.0012061998, 0.036466032, -0.008030085, 0.006042808, 0.01223407, 0.013705128, 0.02539936, 0.0076656947, -0.006440938, -0.013563421, -0.013023583, 0.003944189, -0.01692391, 0.022848628, 0.013657892, 0.029637087, -0.0032930097, -0.013394722, 0.0002456683, -0.011485046, 0.0108439885, -0.011478298, 0.022592206, -0.02440066, 0.009825045, -0.008684638, 0.0024849402, 0.02996099, 0.0053680106, -0.019636594, 0.0035089448, 0.020189928, -5.6112276e-06, 0.027423752, -0.022889117, 0.014386673, 0.013421713, 0.010061224, 0.0019805294, 0.014913015, -0.007564475, 0.0055400836, 0.009717078, 0.033253998, 0.0013352546, -0.022119848, 0.027909607, 0.007955858, -0.012186835, 0.0111881355, -0.001125224, 0.024130743, 0.013684885, -0.014629601, -0.0005453204, -0.014589112, 0.014494641, 0.00871163, 0.0037451237, 0.03500847, -0.0021779074, 0.01266594, -0.0004799494, -0.0036169123, 0.016829439, 0.015425861, -0.0036911399, 0.026317086, 0.0013909255, 0.03473855, -0.003947563, 0.004649352, -0.012524233, -0.013745616, 0.015385373, -0.022322286, -0.028071558, 0.019825537, -0.005762767, 0.008718378, 0.01519643, 0.022322286, 0.00027223842, 0.014913015, 0.023901312, -0.023968792, -0.031904403, 0.040784735, 0.018502936, -0.010985696, -0.017085861, -0.026384564, 0.009730574, -0.019609602, 0.008077321, -0.0023617898, 0.013711876, -0.0049024005, -0.011883176, -0.0031647983, 0.0030922575, 0.019150741, -0.016761959, 0.014346185, -0.014130251, -0.004325449, 0.014913015, -0.036627986, -0.013745616, 0.025493832, -0.0038564652, 0.008758866, -0.005644677, -0.007942362, -0.018435456, 0.023226514, 0.0074565075, -0.0127874045, 0.013927811, 0.0036945138, 0.032930095, -0.0023938427, -0.009811549, 0.023982288, 0.024481637, 0.012956103, -0.032498226, -0.003190103, 0.008030085, 0.045292377, 0.049665064, -0.0056379293, 0.0021154888, -0.015209925, -0.005017116, -0.013644396, -0.033092048, -0.0010096651, -0.008192036, -0.032444242, -0.007159597, 0.012551226, -0.008394475, 0.0103446385, -0.011262363, -0.018597407, -0.021553017, -0.0044975225, -0.041081645, -0.018637896, -0.0022335784, 0.016586512, 0.017396268, 0.008441711, -0.033065055, 0.030473836, 0.019704074, -0.005098092, -0.03039286, 0.005995572, 0.02487302, 0.008941061, -0.0078343935, -0.020162936, 0.024157735, -0.00020908166, -0.007125857, 0.015317894, 0.019798545, 0.020972693, 0.025520824, 0.0058504906, 0.010418867, -0.001013039, -0.024036271, -0.009534882, 0.03427969, -0.0076656947, -0.023253506, -0.027491232, -0.0033014447, -0.010790005, -0.0051554497, 0.0119709, -0.009156996, -0.003907075, -0.007152849, -0.016761959, -0.014899519, -0.0246031, -0.008596915, 0.012524233, -0.028206518, -0.020473342, -0.029043265, -0.0058977264, -0.0023752856, 0.00050905003, 0.004160124, 0.008050329, -0.006498296, -0.042323273, -0.003643904, 0.013192282, 0.028611396, 0.028044567, -0.016802447, -0.013536429, -0.017153341, 0.014400169, -0.02696489, 0.021620497, -0.022902612, -0.020432854, 0.0067749624, -0.027113346, -0.020176431, -0.013887323, 0.010823744, -0.002275753, 0.0073148003, -0.018313993, -0.006390328, 0.013394722, -0.0042377254, -0.012355534, 0.02823351, 0.016249113, -0.02709985, -0.003971181, -0.00072709384, 0.020810742, -0.011977647, 0.0028290867, 0.021040171, -0.0002747689, 0.018705374, 0.005769515, 0.0032727658, -0.013873828, -0.0007334201, -0.0074700033, 0.011734721, 0.04283612, -0.0026772574, 0.0071326047, -0.0021762205, -0.0072743124, -0.014224722, -0.018610902, -0.020594805, 0.010756265, 0.0056109377, -0.011282607, -0.0065522795, -0.0113770785, -0.0022453873, -0.031364568, 0.006707483, -0.013172039, 0.027936598, -0.005236425, -0.015709275, -0.025115946, -0.012274559, -0.02148554, 0.0019569115, -0.008873581, -0.0056885392, 0.004575124, -0.0051993113, -0.018313993, -0.018138545, 0.013401469, -0.024657084, -0.026775947, 0.025628792, -0.0022167084, 0.01519643, 0.0058572385, 0.026290093, -0.009885777, -0.036493026, -0.007942362, -0.014076266, -0.0003595403, 0.020918708, 0.033227008, -0.018044073, 0.041351564, -0.0030635786, 0.025048466, 0.019069765, -0.009926265, 0.007071873, 0.0049597584, -0.0070313853, 0.0054422384, 0.00046181423, 0.010870981, 0.008509191, 0.0312566, 0.002034513, -0.001832074, 0.018502936, -0.027410256, -0.00964285, -0.014143746, -0.024846029, -0.028179524, 0.010013988, -0.018340984, 0.02430619, -0.03716782, 0.0045278883, 0.003890205, 0.0067817103, -0.0039846767, -0.0134756975, 0.019744562, -0.0024174605, -0.01074277, -0.01419773, 0.008603662, -0.010256915, -0.009771061, -0.029124241, -0.005266791, 0.04534636, 0.003883457, -0.001599269, 0.01742326, 0.007625207, 0.00081903493, 0.0060799215, -0.0036945138, -0.01792261, -0.012591713, -0.008495695, -0.017598707, -0.00821228, -0.056143116, -0.0053983764, -0.014737568, 0.0037113838, -0.02287562, 0.023847328, -0.011026184, -0.02823351, -0.020324888, 0.029367168, 0.04270116, 0.004642604, 0.008320248, 0.03185042, -0.023105051, 0.02014944, 0.01642456, 0.0021897166, -0.010202931, 0.030068956, 0.02437367, -0.04402376, -0.006410572, 0.024225214, -0.0012500616, -0.0090557765, -0.018556919, 0.019596107, -0.0065624015, -0.00010923277, -0.01851643, -0.01951513, -0.02480554, 0.01942066, -0.01842196, 0.020095456, 0.00069124525, 0.0055164658, -0.0103108995, 0.026195621, 0.013968299, 0.01705887, -0.0063026045, -0.0020243912, -0.0070583774, 0.007793906, 0.010067972, 0.003183355, -0.004349067, 0.02274066, -0.015115454, 0.014467649, -0.00974407, -0.0010121956, -0.0008688012, -0.0049428884, -0.01606017, 0.02467058, -0.032309283, -0.014656592, -0.0007283591, -0.025183426, 0.00600232, -0.002714371, 0.0040386603, -0.0024056516, -0.008104312, -0.0022723791, 0.029933998, -0.002540611, 0.00059972587, 0.004102766, -0.017760659, -0.0037721156, 0.00137996, -0.025952695, -0.011397323, -0.022835132, -0.013037079, 0.007308052, 0.030905705, 0.035035465, -0.003223843, -0.021458546, 0.017153341, 0.024657084, 0.0008932626, -0.008333744, 0.02539936, -0.00056514255, -0.02314554, -0.017936105, -0.007287808, -0.013462202, 0.018637896, -0.021984888, -0.018152041, -0.008138052, -0.008947809, 0.008482199, 0.021040171, -0.02101318, -0.01699139, -0.00598545, 0.0075374832, -0.019771554, -0.009690085, 0.017747162, -0.00320866, -0.025912207, 0.031445544, -0.021930905, 0.0048686606, -0.026330581, -0.008219028, 0.00043313537, -0.014602608, -0.019231716, -0.013145046, 0.017747162, -0.010978948, -0.008671142, -0.0058437423, -0.007078621, -0.013050575, 0.021620497, 0.022538222, -0.025790744, 0.00059930416, 0.008542931, 0.009852037, -0.008515939, -0.011221875, -0.025628792, -0.018948302, 0.0014423787, 0.0003711384, -0.010081468, -0.017693179, 0.032039363, 0.004767441, -0.030446842, -0.045292377, -0.026060663, -0.029583102, -0.015965698, -0.015115454, -0.0068154503, 0.026020175, 0.018826839, -0.020473342, 0.026978387, -0.0038126034, 0.03068977, 0.0010940147, -0.00021593506, -0.02187692, -0.032444242, -0.008833094, 0.0016397568, -0.016046675, 0.00781415, 0.018138545, -0.00050314557, -0.002744737, 0.03544034, -0.0085564265, 0.009771061, -0.01193716, 0.017355781, 0.033874813, 0.021620497, 0.0026452045, -0.021269603, 0.011296103, 0.0094674025, 0.010229924, -0.022308791, 0.0015182934, -0.0002180438, 0.013914315, -0.009393175, -0.0065927673, 0.010479598, -0.018880822, 0.007793906, 0.024765052, 0.0140087865, -0.030446842, 0.020540822, 0.0067648403, -0.008003093, -0.02101318, 0.02021692, -0.0056581735, -0.015169438, 0.03328099, -0.020878222, -0.017450253, -0.02723481, -0.0042917095, -0.01203838, 0.00069166697, -0.015830738, -0.017301796, -0.009150248, 0.013995291, 0.012679436, -0.04275514, 0.0019400415, -0.02373936, 0.01110716, -0.01985253, 0.0032862616, 0.0011387201, 0.0014643096, 0.044509616, 0.026573509, 0.0077871582, -0.0019501635, 0.024967492, 0.009865533, -0.013489193, 0.01938017, 0.22997084, -0.024265703, 0.013043827, 0.023779849, -0.007962605, -0.026560012, 0.015736267, 0.010877728, -0.024886515, -0.01765269, 0.008515939, 0.006208133, -0.017045373, -0.0028830704, 0.017571716, -0.0058808564, -0.032444242, -0.031337574, 0.007152849, -0.018287001, -0.017571716, -0.01100594, -0.011910168, -0.011390574, 0.022362774, 0.019879522, -0.0032390258, -0.0042141075, 0.018152041, 0.022268303, -0.012362282, -0.013536429, 0.016249113, 0.0063363444, -0.019609602, 0.00018040277, 0.014130251, 0.008684638, 0.039894003, -0.0013740555, 0.03452262, -0.016937407, -0.0069571575, -0.012922363, 0.0040724003, 0.009352687, -0.0140087865, -0.014953502, -0.015533828, 0.008893825, -0.018880822, 0.0068424423, 0.009986997, 0.0046358556, -0.022335783, 0.01007472, 0.008974801, 0.0006735318, -0.017679682, -0.0070043933, 0.002071627, -0.0003863213, 0.019434156, -0.0006165958, -0.023455946, 0.010499842, -0.006464556, 0.007308052, 0.0138198435, -0.016505536, -0.009993744, -0.018246513, -0.024279198, 0.0030112818, -0.039651074, -0.01678895, 0.030338876, 0.027882615, 0.017180333, 0.018044073, 0.005000246, -0.02353692, 0.027909607, -0.011424314, -0.03068977, -0.036061157, 0.019528627, -0.017531227, 0.011667241, -0.01612765, 0.002972481, 0.00052676344, -0.015547324, 0.0035696765, 0.016694479, 0.0076319547, 0.021161636, 0.0047235796, -0.0096563455, -0.016829439, -0.0021762205, 0.061325558, 0.018894318, -0.0040184166, -0.014670088, 0.0017139845, -0.013246266, -0.00557045, 0.013981795, 0.0032879487, -0.008704882, -0.017531227, 0.012024883, -0.00960911, -0.012409518, 0.00964285, 0.010830493, -0.006241873, 0.016100658, -0.0003279092, -0.022970092, -0.019609602, 0.008003093, 0.017072365, 0.00553671, -0.025223915, -0.019393668, -0.006616385, -0.0005866517, -0.0073890276, 0.010688785, -0.012969599, 0.02437367, 0.01851643, -0.018071065, 0.007078621, -0.0016886797, -0.0010965453, 0.012247566, 0.0064949216, 0.0024765052, -0.023658386, 0.009541631, -0.010790005, -0.0035730505, -0.0066096373, -0.006191263, 0.00412301, 0.0117549645, -0.014751064, -0.022457246, -0.02280814, -0.021782449, 0.0009936386, 0.0072338246, 0.020878222, -0.0023938427, -0.03068977, -0.0192857, 0.0072000846, -0.041837417, 0.0088668335, 0.011248867, -0.01639757, -0.024819035, -0.0044941483, -0.17242414, 0.011080167, 0.0006284048, -0.03414473, -0.0035494326, -0.0069908975, 0.010499842, 0.0035595545, -0.033874813, 0.0005077848, 0.015668787, -0.01705887, -0.0134419575, -0.011910168, -0.038463432, 0.015250414, -0.0033739852, -0.0028712614, 0.021647489, 0.01765269, 0.028800339, -0.015979195, 0.013941308, 0.012625453, -0.013192282, -0.021175131, 0.008488947, 0.02882733, 0.0011884863, -0.01103968, -0.002543985, -0.009818297, 0.021930905, -0.009501142, 0.01699139, 0.021323588, 0.013927811, 0.014764559, -0.013239518, 0.0109654525, 0.024049766, 0.018313993, -0.0017207325, -0.0018911187, 0.003453274, 0.0058336207, -0.0038733352, -0.026411558, -9.325906e-05, -0.015385373, -0.0030990054, -0.004642604, 0.032282293, -0.0017148281, -0.008111061, 0.021714969, 0.029205216, 0.0021323587, 0.006339718, -0.008421468, 0.008664394, -0.032282293, 0.020864725, -0.0019923383, -0.0109654525, -0.029799039, -0.014616104, -0.005894352, -0.020972693, 0.017153341, -0.0065117916, -0.010250167, 0.014535129, -0.009136752, 0.014980495, 0.032012373, -0.006930166, -0.0131652905, -0.0042883353, 0.011694233, -0.031310584, 0.03185042, -0.015493341, 0.023982288, 0.005752645, 0.00686606, 0.008536182, 0.00781415, -0.029853022, -0.0023347978, 0.014278706, -0.023860823, 0.009298704, -0.018313993, 0.010479598, 0.0037147577, 0.024522126, 0.017139845, 0.008455208, -0.011019436, -0.0016844622, -0.0013040453, -0.0008472921, 0.0054186205, 0.012915616, -0.008259516, -0.010951956, -0.010945208, 0.039516117, -0.027126841, -0.013570169, -0.011849436, 0.024387166, 0.007517239, -0.018961797, 0.036304083, -0.02041936, -0.028179524, 0.0102771595, -0.008961305, 0.051878396, -0.0099667525, -0.010823744, 0.028800339, -0.020689277, -0.035197414, -0.110072896, -0.007901873, 0.0070246374, 0.00096242933, -0.0026198996, 0.04075774, -0.007976101, 0.009636102, -0.007733174, 0.03168847, -0.010918217, -0.029637087, 3.91435e-06, -0.01898879, -0.0030585176, -0.011653746, 0.004814677, -0.011289354, -0.022592206, 0.03946213, -0.0013107933, -0.02467058, 0.018084561, -0.012180087, 0.024333183, -0.010432363, -0.03252522, 0.01944765, 0.022133343, -0.013495941, 0.026829932, -0.019798545, -0.0011117281, -0.013151795, -0.017504236, 0.013536429, -0.014778055, -0.016208624, -0.013981795, -0.025547817, -0.004855165, 0.009170492, 0.00015562506, -0.016667487, -0.022767652, -0.021714969, -0.01765269, 0.02666798, -0.0008177697, -0.023024075, -0.032336276, -0.01240277, -0.007942362, 0.004807929, 0.016411064, 0.026546516, -0.011478298, -0.009197484, 0.004740449, -0.02307806, -0.022187328, -0.012429762, -0.02064879, 0.0076521984, 0.01758521, -0.0060765473, -0.008360735, 0.002321302, 0.0006140653, -0.008785858, -0.0029404282, -0.0026502653, -0.008050329, 0.02696489, -0.028557412, -0.008138052, -0.009636102, -0.0156283, -0.0065657753, -0.009271711, 0.00684919, -0.0069504096, -0.004193864, -0.010830493, -0.022200823, -0.0030264647, -0.0148860235, -0.0019113626, -0.031229608, -0.001493832, -0.021553017, 0.028260501, -0.01269968, -0.006184515, -0.014818544, 0.016829439, -0.02135058, -0.007328296, 0.02071627, -0.0070246374, 0.0008380136, 0.014494641, -0.04226929, 0.019177733, 0.0068593123, -0.018691879, -0.0012584965, -0.010317647, 0.021930905, -0.0013048889, -0.023280498, 0.012929112, 0.009096264, 0.012848136, -0.019758059, 0.0059314664, -0.018840333, -0.013651145, 0.007517239, -0.017004887, 0.011775209, 0.030635787, -0.003930693, -0.0027869118, 0.020014482, -0.00079668226, 0.0016836186, 0.011302851, -0.031337574, 0.027693672, -0.022511229, -0.022862125, 0.021323588, -0.029286193, -0.010884477, 0.00036249254, -0.00013907146, 0.016181633, -0.009345939, 0.03684392, 0.03571026, 0.014508137, -0.024090255, -0.033227008, 0.0060495557, -0.0021812816, 0.0040386603, 0.016815942, -0.025831232, -0.019542122, 0.003896953, -0.0037518716, 0.033361968, 0.016168138, -0.016546024, -0.014521632, -0.021593506, -0.023186026, 0.007841142, -0.0027329281, 0.0007190806, -0.0071663447, 0.0405688, 0.03252522, 0.00137996, -0.030176925, 0.007962605, 0.007577971, -0.018192528, 0.010601061, 0.014872527, -0.0048855306, -0.0015174499, 0.0092447195, 0.002080062, 0.01692391, 0.0035055708, 0.0005832777, 0.021903912, 0.028989282, -0.0058673606, 0.030068956, 0.0039644325, -0.025952695, -0.0017409764, -0.019973993, 0.023442449, 0.024117246, -0.0017063931, 0.007895126, -0.009399923, 0.0097778095, 0.004160124, 0.011147647, -0.010398623, -0.0037181317, -0.0041736197, -0.016276104, -0.030365868, 0.02996099, 0.013205779, 0.017490739, 0.025763752, -0.0036371562, -0.012045127, -0.03039286, -0.029421153, 0.018435456, -0.029313184, -0.05017791, 0.028260501, 0.032363266, -0.009352687, -0.019231716, -0.01330025, 0.002714371, -0.015520332, 0.014319194, 0.0012778969, -0.009332444, -0.025655784, 0.03041985, 0.009251468, 0.00070010195, 0.03876034, -0.0065590274, 0.020068465, -0.010317647, 0.015547324, -0.020122448, 0.020905213, 0.013563421, -0.0066096373, 0.0008397006, 0.009406671, 0.004143254, -0.012261063, 0.020918708, -0.0063059786, 0.02839546, 0.010101712, 0.07595516, 0.00686606, 0.008023337, 0.018813342, -0.01592521, 0.011923664, 0.02430619, 0.010648297, -0.014805048, -0.003439778, 0.009082768, 0.013097811, 0.004598742, -0.023469443, -0.0049631326, -0.021040171, -0.009730574, 0.023496434, -0.002731241, -0.0040993923, 0.003667522, 0.019258708, 0.015277405, -0.01323277, 0.00082198717, 0.010553826, 0.017234316, -0.0014052648, -0.0037923595, -0.015641795, -0.0022217694, 0.01672147, -0.034603592, -0.0037181317, 0.0038767091, -0.0020378872, 0.00058201246, -0.016883422, 0.006457808, -0.016276104, -0.02543985, 0.016357081, -0.010668541, -0.0047202054, -0.005546832, -0.009892525, -0.00061322184, -0.014035779, -0.03514343], index=0, object='embedding'), Embedding(embedding=[-0.008914787, 0.02012447, 0.00023190839, -0.009403641, 0.0074821752, 0.0068133962, -0.010055446, 0.014203912, -0.00979744, -0.0125879785, 0.0037105342, 0.042530257, 0.0041892035, 0.008711099, -0.001783976, 0.01127758, 0.018427063, 0.013796533, 0.0040398315, -0.02497227, -0.02070838, -0.014706344, -0.018617172, 0.003355776, -0.012920671, -0.0031334155, 0.031340946, -0.029059628, -0.011508428, 0.0033642629, -0.007230959, -0.01681113, -0.012173811, -0.0032692081, -0.018929496, -0.0137761645, -0.0073124343, -0.028760884, 0.01275093, -0.008283352, 0.016009953, 0.006871108, 0.0065927333, -0.008758626, -0.0057745823, 0.014869295, 0.007339593, -0.0032114962, -5.458228e-05, -0.010225187, 0.022419367, 0.019866465, -0.026085768, -0.0008457336, -0.020586167, 0.010001129, -0.020749116, 0.034599967, -0.012411448, -0.027511591, 0.0014003618, -0.013518158, -0.0037444823, 0.005061671, -0.020287422, -0.011202894, -0.023342757, -0.0030723088, 0.026167244, 0.0067556845, 0.038347844, 0.03880954, -0.01016408, -0.018454222, 0.009023421, -0.005526761, 0.007903133, -0.010041866, 0.0018977023, 0.020246685, 0.0022236046, -0.008208666, -0.0091727935, 0.035441883, -0.0009895889, -0.0011423557, -0.0010031682, 0.025406806, -0.0076994435, -0.009844967, 0.021713246, 0.019336874, 0.021210812, 0.017870313, -0.017856734, 0.0029365162, -0.02087133, 0.0067455, 1.7663653e-05, -0.00461695, 0.0048477976, -0.006684393, -0.008480251, -0.016091429, 0.0059443233, 0.0013112478, 0.03174832, -0.004837613, 0.00461695, 0.013294101, -0.010972046, 0.0073871203, 0.004939458, -0.021726824, 0.01644449, -0.012825616, 0.031558212, -0.011508428, 0.0053060977, -0.016838288, 0.03009165, -0.0046712672, 0.016702496, -0.0016074455, 0.013273732, 0.003676586, -0.0053196773, -0.02474142, -0.023804452, -0.028679408, -0.003489871, 0.040221784, 0.001875636, -0.01773452, -0.017517252, 0.014380442, -0.0070680077, 0.002836369, -0.013871219, -0.007923502, 0.014298966, 0.017666625, -0.026099348, -0.008717888, -0.0010659723, -0.0054215216, -0.007930291, 0.024171092, 0.0041620447, -0.009532644, 0.015588996, -0.020681221, -0.012683034, 0.0037750357, 0.010972046, 0.0184678, 0.0098517565, 0.03144958, -0.026764732, -0.004287653, -0.009722753, 0.007224169, 0.0019571115, 0.029331213, 0.019336874, 0.022663793, -0.0058662426, 0.004145071, 0.008385196, -0.013368786, -0.0103474, 0.015914898, -0.025637653, -0.0028720147, -0.006782843, -0.0048613767, 0.033893846, 0.005655764, -0.00489872, -0.027144952, -0.0013706571, 0.014706344, -0.006355096, 0.007964239, -0.009030212, -0.017829575, 0.030308919, -0.01294104, 0.011304738, -0.02106144, 0.026085768, 0.022840325, -0.01053751, 0.021523135, -0.64398307, 0.002561389, 0.042937636, -0.01867149, 0.013382366, 0.0030247814, 0.0072377482, 0.0015862279, -0.0037750357, 0.015738368, -0.003240352, -0.021495977, -0.014529814, -0.017150613, 0.009525854, -0.01590132, -0.014882875, -0.015317411, 0.012153442, 0.005509787, -0.03492587, 0.017028399, -0.0024544522, 0.015303832, 0.007081587, 0.015072985, 0.00868394, -0.041688345, -0.006636866, 0.010883781, -0.033921003, 0.02145524, -0.0035917156, 0.010571458, 0.044078294, -0.006966163, -0.027620226, 0.0076043885, -0.0011016179, 0.02957564, -0.017340722, -0.0026360748, 0.023818031, -0.0036324535, -0.00470861, 0.015303832, 0.0020114286, 0.007427858, 0.0050922246, -0.0030604268, 0.0043317857, 0.015303832, -0.012425028, -0.01423107, 0.0199751, 0.0016133864, 0.039515663, -0.023342757, 0.0060393782, -0.015942058, -0.011386214, 0.014366862, -0.023410654, -0.026832627, -0.01645807, 0.004199388, -0.011460899, 0.043127745, -0.005041302, -0.032834664, 0.01883444, 0.016838288, -0.0043555493, 0.006416203, -0.008351248, 0.021020703, 0.025664812, 0.02163177, -0.009084528, 0.0029789512, 0.006759079, -0.018902337, -0.012859564, -0.0020284026, 0.018399904, 0.011460899, -0.034844395, 0.0020131262, -0.003048545, -0.00397533, -0.0066674193, 0.020178787, -0.030906407, -0.05160121, -0.019554142, 0.016580284, 0.0012340157, 0.021333026, 0.027375799, 0.01293425, 0.012431817, -0.010978836, 0.0055505247, -0.009715964, 0.015032247, 0.018101161, -0.0019740856, 0.0011381122, 0.007977818, -0.01901097, -0.021903355, -0.025216695, -0.0008032984, -0.007889553, -0.024143934, -0.039026808, 0.023003275, 0.024985848, -0.0120787565, 0.0019027946, 0.007930291, 0.0006187053, 0.012961408, 0.0065723644, -0.0005406245, 0.003584926, -0.0034542256, -0.027212847, -0.01865791, 0.016376594, 0.02125155, -0.0062736203, 0.02070838, -0.016892606, 0.0061819605, 0.010849833, 0.01628154, -0.021468818, -0.01164422, -0.012581189, -0.019635618, 0.00997397, 0.017041977, -0.020857751, -0.020721959, -0.009478327, -0.021930514, 0.0094308, 0.012615138, 0.009064159, -0.005720265, -0.0016218735, -0.021346604, 0.0049632215, 0.0041892035, -0.010924519, -0.0034830815, -0.034898713, -0.009532644, -0.0425031, -0.0045388695, 0.026873365, -0.029249737, 0.012655875, -0.00021206994, -0.014747082, 0.0010846438, 0.010775147, -0.038320687, -0.007319224, 0.025230275, -0.044675782, 0.010225187, 0.021305867, 0.009213531, 0.009464748, -0.00059494155, -0.008935156, -0.005856058, -0.021658927, 0.021835458, -0.00554034, 0.001497114, 0.0038157734, 0.003948171, -0.00978386, 0.014652027, 0.006704762, -7.213985e-05, 0.0002928029, 0.011297949, -0.013328048, 0.00071163836, 0.009369693, 0.009064159, 0.01884802, -0.0054045473, 0.032019906, -0.007998187, 0.009654857, 0.00885368, -0.001921466, -0.016906185, 0.016132167, 0.018454222, -0.005431706, -0.00127051, -0.015168039, 0.0347086, 0.004155255, 0.005632, -0.0060936953, -0.029820066, -0.0017585148, 0.01792463, 0.04155255, 0.0039719352, 0.0077333916, -0.0036731912, 0.01182075, -0.030553346, -0.006833765, 0.032807503, -0.010489983, -0.013280521, -0.0049055098, 0.002507072, 0.009023421, 0.028299188, -0.0076994435, 0.015806265, 0.023369916, -0.008629623, 0.023410654, 0.018590014, -0.012071967, 0.025393225, 0.014937191, 0.039569978, 0.0016278144, 0.011990491, 0.025922816, 0.0032030093, -0.013491, 0.013973064, -0.004070385, 0.023885928, 0.0034151853, -0.005261965, -0.0046508983, -0.015602576, 0.0051295673, 0.013796533, -0.005132962, 0.026819048, -0.0029331213, 0.010666513, 0.02497227, 0.0047731115, 0.03568631, -0.007115535, -0.01052393, 0.006429782, -0.019296136, 0.011610271, -0.010856623, 0.003659612, -0.0069423993, -0.016974082, 0.02198483, -0.024062458, -0.029521322, 0.01755799, -0.039162602, 0.01867149, 0.016390173, 0.017232088, 0.0068846876, 0.022297153, -0.0033337097, 0.006535021, -0.026017873, 0.030173127, 0.009213531, 0.009641279, -0.013789744, -0.022392208, 0.018223373, -0.022297153, -0.004253705, 0.008602465, -0.0004370826, 0.020572586, -0.016892606, -0.013871219, 0.01405454, 0.026194403, -0.0038293528, 0.014516234, -0.009614119, -0.008038925, 0.012187391, -0.02308475, -0.0027396167, 0.05491455, -0.013205836, -0.008038925, 0.00046211938, -0.0068269754, -0.008038925, 0.0031147439, 0.0047221896, -0.016525965, 0.01405454, -0.0056252107, 0.006874503, -0.008378407, -0.004097543, 0.0155211, -0.0005283183, 0.01682471, -0.04470294, 0.006137828, 0.024890793, 0.02680547, 0.05686996, -0.027552329, -0.003948171, -0.01146769, -0.0012637203, -0.009125266, -0.037044235, -0.012649085, -0.01182075, -0.024252567, -0.019309714, 0.0016532756, 0.0021896565, 0.008344458, 0.021129336, -0.011637431, -0.024483414, -0.0118886465, -0.00323526, -0.013599634, 0.00065435085, 0.009763491, 0.009736333, 0.011494848, -0.031558212, 0.009227111, 0.014733503, -0.013613213, -0.02384519, 0.002108181, 0.013959484, 0.011515217, 0.003605295, -0.014298966, 0.014869295, 0.0018128318, 0.0014606197, 0.021753984, 0.0039821197, 0.0057440293, 0.028407823, 0.0005423219, 0.010130132, 0.020341739, -0.013389155, -0.014502655, 0.03473576, -0.025556177, -0.027579488, 0.018345587, -0.007332803, -0.038728066, -0.019526983, -0.0068609235, -0.0006433177, -0.009213531, -0.001565859, -0.0065689697, -0.024347622, -0.0027498011, -0.016851868, -0.00231187, -0.02639809, 0.0031537844, -0.022025568, -0.005720265, 0.0032929718, 4.161302e-05, 0.004209572, 0.0010243858, -0.008901209, -0.0388367, -0.02160461, 0.00062422187, 0.037777517, 0.021333026, -0.017815996, -0.0029246341, -0.033730894, -1.9095842e-05, -0.014285387, 0.026764732, -0.036501065, -0.017829575, 0.010367769, -0.01776168, -0.019268977, -0.007319224, -0.0006407716, 0.012465766, 0.004841008, -0.001775489, 0.006338122, 0.01108747, 0.010184449, 0.01311078, 0.008778995, 0.0019027946, -0.02753875, 0.0039447765, -0.012852774, 0.010089394, -0.029412689, -0.0009658252, 0.014448338, -0.007631547, 0.011983701, 0.008453093, -0.0057304497, 0.0120787565, 0.0049768006, -0.0048002703, -0.0025223484, 0.032182857, -0.002108181, -0.019472666, 0.011101049, -0.0011236842, -0.017028399, 0.0041348864, -0.012947829, 0.023532866, 0.004657688, -0.01238429, 0.021482397, -0.0058832164, -0.0155211, -0.019323295, 0.0018399904, -0.013769375, 0.033513628, -0.012947829, -0.010822674, -0.024510574, -0.007190221, -0.030797774, -0.016295118, -0.022025568, -0.016729655, 0.0020063363, -0.0013010633, -0.008052505, -0.03085209, -0.009654857, -0.023926666, -0.021753984, 0.038510796, -0.0051058037, 0.023818031, -0.0069729527, 0.013008936, -0.0010837951, -0.024429098, -0.032291494, -0.026099348, -0.014502655, 0.025026586, 0.029222578, -0.007502544, 0.06317074, -0.009457958, 0.0072920653, 0.027457274, 0.004857982, 0.008778995, 0.011257211, -0.010585037, -0.02235147, -0.00037788547, 0.027253585, -0.010632564, 0.009858547, -0.025759866, 0.020409636, 0.016974082, -0.025759866, 0.009437589, -0.0037784306, -0.013205836, -0.03473576, 0.005122778, 0.006022404, 0.010517141, -0.041362442, 0.0054758387, 0.02624872, 0.008201876, 0.03275319, 0.006392439, 0.040221784, -0.0036120845, -0.010571458, -0.0056795278, -0.020545429, -0.015453204, -0.009485117, -0.03698992, 0.0040330417, 0.023179805, 0.0016804341, 0.009858547, -0.0057236603, 0.006338122, 0.00122468, 0.016091429, 0.037234344, -0.015765527, -0.0077537606, -0.013382366, -0.041688345, -0.032019906, -0.05456149, -0.0081543485, 0.0003193249, 0.014814978, -0.02015163, 0.016675338, -0.014570552, 0.012581189, -0.029521322, 0.024252567, 0.039162602, -0.012703403, 0.024293305, 0.02772886, 0.004837613, -0.012363921, 0.025583336, 0.014665606, -0.0034610152, 0.017612306, 0.020015838, -0.0112096835, -0.035930734, 0.012031229, -0.010449245, -0.00867715, -0.02384519, 0.012995357, 0.00058687886, 0.011963333, 0.005951113, -0.008120401, -0.007319224, 0.003948171, -0.036745492, 0.04024894, -0.0042570997, -0.006433177, -0.027647384, 0.05113951, -0.006266831, 0.026506726, -0.0076994435, 0.0012144955, 0.003713929, 0.00960733, -0.009390062, 0.024836475, -0.00011457505, 0.02474142, -0.01682471, 0.022894641, 0.0035238194, 0.007862395, -0.025759866, -0.005611631, -0.012112704, 0.026683256, -0.04193277, 0.0013876312, -0.009437589, -0.006704762, 0.018386325, -0.007427858, -0.014203912, -0.008262983, 0.0061072744, 0.017069137, 0.013905168, -0.008208666, -0.000810088, 0.012296025, -0.020654062, -0.007964239, -0.01127758, -0.021360185, -0.0077741295, -0.045789283, -0.027633805, 0.008663571, 0.004762927, 0.039162602, -0.004379313, -0.00776055, 0.019608459, 0.01698766, -0.009878916, -0.00051898253, 0.013463842, -0.0023135673, -0.009070949, 0.00036748886, 0.00016793732, -0.0060903006, 0.01421749, -0.011929384, -0.01200407, -0.01478782, -0.019920781, -0.0032539314, 0.0074957544, -0.0015548258, -0.02050469, -0.0016804341, 0.022663793, -0.009749913, -0.033187725, 0.02384519, -0.008772206, -0.00582211, 0.017829575, -0.024727842, -0.0011058614, -0.0042435206, -0.00850062, 0.010632564, -0.023818031, 0.005014144, -0.0108905705, 0.016716076, -0.018386325, -0.015670473, 0.009037001, -0.015235935, -0.0009352719, 0.024442676, -0.0051261727, -0.025990713, 0.020491112, 0.0036460327, 0.0037342978, -0.00924069, -0.019296136, -0.013844061, 0.002383161, -0.013341628, -0.010972046, -0.013049674, -0.022120623, 0.032074224, 0.028950993, -0.022229256, -0.024157513, -0.0029212395, -0.04508316, -0.0030672166, -0.0035170296, -0.013260152, 0.006670814, 0.021658927, -0.0033506837, 0.04652256, -0.020382477, 0.017395038, 0.0061276434, 0.013687899, -0.0064976783, -0.011881857, -0.015222357, -0.0055131814, -0.008975894, -0.00086228334, -0.0033880267, 0.017299984, 0.008711099, 0.043915343, 0.011264, 0.0034134877, -0.006596128, 0.013891588, 0.02604503, 0.02531175, 0.005061671, -0.016172905, -0.006986532, -0.0033303148, -0.0022931984, -0.024143934, -0.013918747, -0.002778657, 0.019554142, 0.0035747415, -0.019146765, -0.0072377482, 0.0015955636, -0.00849383, 0.023722976, -0.008093242, -0.014312546, 0.015168039, 0.02125155, 0.0033421966, -0.03087925, 0.018372746, -0.022650214, 0.013986643, 0.008487041, -0.022120623, -0.008385196, -0.029276896, 0.0039990935, -0.023044012, 0.013769375, -0.0155211, -0.0018315034, -0.011196104, 0.014665606, 0.018359166, -0.022786006, -0.012357132, -0.019119605, 0.016838288, -0.013280521, 0.005438496, -0.0070136907, -0.015276673, 0.025284592, 0.007814867, -0.000983648, 0.016512387, 0.0056286054, 0.00048885355, -0.005241596, 0.024116775, 0.22020136, -0.029928701, 0.011345476, 0.021509556, 0.0030994674, -0.010327031, 0.0028635275, -0.001347742, -0.009104897, 0.0016643087, 0.019350452, 0.0061683813, 0.016200064, -0.0063449116, 0.016729655, 0.011134997, -0.026303036, -0.03788615, 0.00397533, 0.00776055, -0.0021523135, -0.021957671, 0.0010142013, -0.022283575, 0.02127871, -0.015806265, -0.015955636, -0.006711552, 0.006932215, 0.035251774, -0.01755799, -0.012581189, 0.03275319, 0.0027752623, -0.024048878, 0.019893624, 0.01610501, -0.027796756, 0.007054428, -0.0018739386, 0.034111112, -0.015005088, -0.0014580736, -0.012221338, -0.0051125935, 0.008161139, -0.01831843, -0.008697519, -0.0014241254, 0.01865791, -0.0044064717, -0.006239672, 0.025352487, 0.03272603, -0.012601558, -0.004416656, 0.00868394, 0.00026203739, 0.019907203, -0.0075636506, -0.0045863967, 0.017883893, 0.02662894, -0.0040330417, -0.0046678726, 0.010639355, 0.005890006, 0.01311078, 0.018454222, -0.014108856, 0.029059628, -0.02089849, -0.030417554, 0.009199952, -0.025379647, -0.031015042, 0.05059634, 0.025949975, 0.025040165, 0.018345587, -0.004229941, -0.0042570997, 0.017585149, -0.014584131, -0.02288106, -0.033187725, 0.0039787246, 0.0054351008, 0.006080116, -0.014027381, 0.002087812, 0.0019163738, 0.0030502423, 0.009885705, 0.018820861, -0.012506504, 0.006857529, -0.00090386986, -0.005065066, 0.01698766, -0.028896676, 0.07740181, 0.025909238, -0.018250532, 0.0113794245, 0.019160343, -0.015005088, -0.013185467, 0.03954282, -0.012533662, 0.0052382015, -0.021088598, 0.017041977, -0.010938098, -0.006820186, 0.015806265, 0.04540906, -0.014176752, 0.0007044244, 0.008487041, -0.0125879785, -0.012071967, 0.013192256, 0.016906185, -0.011664589, -0.0050345124, -0.02531175, -0.011800381, -0.012343552, -0.0146248685, 0.025203116, -0.00924069, 0.02567839, -0.0047799014, 0.0063041737, 0.014814978, 0.013368786, 0.014285387, -0.008616044, -0.00033396506, 4.5352623e-05, -0.029792909, -0.012486135, -0.00041819894, -0.0009794045, -0.0065723644, -0.007862395, -0.0012577794, -0.005862848, -0.012282445, -0.028733725, -0.014896454, -0.017164191, 0.013137939, 0.0024731236, -0.0012806945, -0.015466783, -0.038103417, -0.0034593178, 0.02605861, -0.03158537, -0.009715964, 0.01607785, -0.015005088, -0.016797552, -0.003880275, -0.17348869, -0.0034661074, 0.008459882, -0.038347844, 0.019486245, 0.004287653, 0.030037334, -0.0038225632, -0.018983813, -0.013239783, 0.0004837613, 0.019526983, -0.042177197, 0.0034864764, -0.022229256, 0.0024799134, -0.0041111223, -0.023003275, 0.020450374, 0.007407489, 0.035441883, -0.042285834, -0.0013969669, -0.028597932, -0.020925649, -0.020762697, 0.013640372, 0.02050469, 0.011325107, -0.019825727, -0.0055165766, -0.012167022, 0.027674543, -0.018074002, 0.012465766, 0.010272714, 0.026873365, 0.01681113, 0.0044438145, 0.014000222, 0.024605628, 0.026465988, 0.0075364923, -0.010265924, -0.02015163, 0.014556972, 0.01423107, -0.0017975552, 0.02753875, 0.0043216012, 0.0144076, -0.031368103, 0.013973064, 0.013212625, 0.013905168, 0.011542375, 0.020436794, 0.00692882, -0.007244538, -0.014747082, 0.023424232, -0.011732485, 0.0124182375, 0.007278486, -0.01347742, -0.006246462, -0.0124386065, -0.009335745, -0.0017169283, 0.0052008587, -0.016172905, -0.00017483304, -0.007726602, 0.011386214, 0.008466672, 0.027946128, -0.015616155, -0.013640372, 0.009335745, -0.036799807, -0.014570552, 0.031721164, 0.0029246341, 0.001657519, -0.00040971188, -0.003581531, 0.022772428, -0.017137032, -0.008975894, -9.022785e-05, 0.026506726, -0.043670915, 0.02122439, 0.0037750357, 0.0009658252, 0.012431817, -0.0039583556, 0.018101161, 0.017313562, -0.0039278027, -0.0044030766, -0.005947718, -0.012296025, 0.026819048, 0.008935156, 0.009308586, -0.017897472, -0.0050888294, 0.021862617, -0.037913308, -0.0077877087, -0.01829127, 0.0062091188, 0.00030680653, -0.011318318, 0.049808744, -0.01571121, -0.011752854, 1.8830622e-06, -0.01478782, 0.06078079, 0.0040398315, -0.03123231, 0.013307679, 0.009206742, -0.0033744473, -0.11221905, -0.015127301, -0.0004010127, 0.0052993083, -0.0008147559, 0.039787248, -0.0024917952, -0.006008825, -0.031313784, 0.028353505, -0.008989474, -0.033214882, -0.0054826285, -0.010130132, -0.006799817, 0.007034059, -0.009654857, -0.0061140643, -0.018943075, 0.035496198, -0.020287422, -0.0024884003, 0.03546904, -0.022663793, -0.010150501, 0.021468818, -0.017218508, 0.019323295, 0.016376594, -0.017164191, 0.009532644, -0.012248497, -0.0037784306, -0.013083622, -0.015588996, 0.0017347512, -0.011861488, -0.0010082604, 0.0024985848, -0.019364033, 0.0061208536, 0.00721059, 0.0029093577, 0.006528232, -0.021509556, 7.3333335e-06, -0.015127301, 0.012465766, -0.0029993202, -0.0055742883, -0.03584926, -0.008928367, -0.016688917, -0.014652027, 0.017435776, 0.010231976, -0.01662102, -0.0019095842, -0.008201876, -0.0054079425, -0.05817357, -0.012615138, -0.03256308, 0.0015556746, 0.013497789, -0.01256761, -0.028244872, -0.02270453, -0.024646366, -0.008928367, -0.00637207, 0.01738146, -0.009553013, 0.015670473, -0.03541472, -0.009037001, 0.0016846776, -0.018277692, 0.021333026, -0.01996152, -0.0050888294, -0.028136237, 0.0024425704, -0.006341517, -0.026140084, -0.00022511875, -0.016553124, -0.0037410876, -0.0048138495, -0.018603593, -0.013782954, 0.009199952, 0.017815996, -0.016037112, -0.026588202, 0.027756019, -0.01848138, -0.009641279, 0.0002327571, 0.0094919065, -0.016023533, 0.012615138, -0.03899965, 0.021944093, 0.010938098, -0.031721164, -0.03416543, -0.003389724, 0.005910375, 0.00084021705, -0.003106257, 0.023695817, -0.01662102, 0.0031385077, 0.0044302354, 0.020042995, -0.020015838, -0.021889776, 0.030607663, -0.027918968, 0.0034796868, 0.0369356, -0.0077469707, 0.006986532, 0.006446756, 0.04011315, 0.0038768803, -0.00850741, -0.0068439497, 0.014638448, -0.011807171, -0.012031229, -0.015629735, -0.037913308, -0.005414732, 0.0013231296, 0.0184678, 0.019377612, -0.00905737, 0.026316617, 0.04489305, 0.046794146, -0.021292288, -0.030634822, -0.008826522, 0.009824598, 0.0072920653, 0.003186035, -0.0034406462, -0.012038019, 0.0042604944, -0.0051974636, 0.03416543, -0.0026852996, -0.014760661, -0.021726824, -0.004263889, -0.014502655, -0.0014113949, -0.002133642, 0.017748099, -0.01605069, 0.03734298, -0.026561042, 0.019350452, 0.0055335504, 0.031558212, -0.005065066, -0.013959484, 0.019133184, -0.000536381, -0.013008936, 2.845811e-05, 0.018209795, -0.005727055, 0.014733503, 0.0015302134, 0.009485117, -0.00739391, 0.007930291, 0.006932215, 0.012336763, -0.009009843, -0.0044981316, -0.0036256637, 0.003605295, 0.04046621, 0.01791105, -0.0065859435, 0.014719923, -0.006708157, 0.011922595, 0.01421749, 0.013932326, 0.005414732, 0.009946812, -0.009743122, -0.011155366, -0.01164422, 0.0394885, 0.034871552, -0.00065010734, 0.023451392, -0.0030213865, -0.015249515, -0.04065632, -0.008948736, 0.023057593, -0.01625438, -0.028760884, 0.0059612975, 0.033296358, 0.005465654, 0.0073871203, -0.022473684, 0.0044573937, 0.0011780013, 0.0073803305, -0.0076451264, -0.02843498, -0.02715853, 0.02642525, -0.00039422305, 0.009967181, 0.04062916, 0.0125879785, 0.011637431, 0.008378407, 0.012065177, -0.02089849, 0.013409524, 0.0064535458, -0.0024646367, 0.0031130465, 0.012200969, 0.0066538397, -0.017327143, 0.0006942399, -0.009695595, 0.02127871, 0.030254602, 0.07778203, 0.014380442, 0.010618986, -0.005543735, -0.05024328, 0.00067047623, 0.016200064, 0.0056218156, -0.0052178325, -0.021414502, 0.0101708695, 0.015616155, -0.0020861146, -0.025814183, -0.014380442, -0.00094969984, -0.025719129, 0.017680204, -0.022595897, 0.014719923, 0.009525854, 0.013810113, 0.00888084, -0.0015981097, -0.022460105, 0.00508204, 0.014108856, 0.014108856, -0.016240802, -0.026017873, 3.9385173e-05, 0.012248497, -0.047092892, 0.0044098664, 0.003527214, -0.007115535, 0.0038734854, -0.0062770154, 0.017327143, 0.009145635, -0.007977818, 0.03256308, -0.005384179, -0.009485117, -0.014149594, 0.0031164414, 0.008649992, -0.018169057, -0.00425031], index=1, object='embedding'), Embedding(embedding=[-0.010580079, -0.0030184747, -0.0081748925, 0.011902588, 0.012526155, -0.003313127, -0.0100935595, 0.008346202, -0.022681387, -0.0019512159, 0.006129456, 0.043471515, 0.00046339238, 0.01432148, 0.015746776, 0.023462558, 0.020255644, 0.005807394, 0.0095590735, -0.008257121, -0.02869778, 0.004406082, -0.009017736, -0.025669025, -0.026655769, -0.01906333, 0.023489967, -0.026696883, 0.004769258, -0.00024026167, 0.007681521, -0.011148826, -0.0076335543, -0.00511873, -0.03519384, 0.015308223, 0.0031983496, -0.02979416, 0.019282605, -0.016185328, 0.01124476, 0.006376142, 0.016719814, -0.013697913, 0.02274991, -0.005303744, 0.01400627, 0.0032480294, -0.01751469, 0.010799355, 0.041689895, 0.02458635, -0.052900393, -0.0013927468, 0.0022852696, 0.0019529291, -0.011813507, 0.007935059, -0.0039846604, -0.03672877, -0.0030287532, -0.019611519, -0.00027259634, 0.013040084, -0.028752597, -0.011635345, -0.017076138, 0.013478637, 0.03697546, 0.007948764, 0.012779694, 0.016993908, 0.0023931947, 0.004210789, -0.008387316, 0.00027730735, 0.002951664, -0.010908993, 0.009230159, 0.029437836, 0.0042861653, -0.013334736, -0.02568273, 0.045335364, 0.012067046, -0.025285292, 0.0045259986, 0.01432148, -0.011977965, -0.010401917, 0.0055572824, 0.007105921, 0.022242835, 0.009230159, -0.0175421, -0.014814851, 0.005214663, 0.0054544965, 0.013334736, -0.028560732, 0.0057834107, 0.024654873, -0.035824258, -0.016418308, -0.00966186, -0.0014295783, 0.012663203, -0.018295862, 0.022146901, 0.01006615, -0.015664546, 0.040346827, -0.004275887, -0.017295415, 0.015102651, 0.0057902634, 0.020653082, -0.02657354, 0.0028848532, 0.005303744, 0.018679595, 0.009483698, 0.012560417, 0.0060814894, 0.0121013075, -0.009147931, -0.018405499, -0.013677356, -0.046294697, -0.016089395, -0.0004171388, 0.021187568, -0.0068626613, 0.024928968, -0.02861555, 0.020118596, -0.0069722994, 0.004591096, -0.039634183, -0.009284979, 0.030671265, 0.028889645, -0.015746776, 0.025696434, 0.004066889, -0.0020625673, -0.013519751, 0.018076586, -0.015404156, -0.032507703, 0.0065405993, -0.015116356, 0.0050399275, -0.0017336528, 0.014746328, 0.010107265, 0.0034518878, 0.03143873, -0.013629389, -0.010141526, 0.015294518, 0.019570405, 0.0064343875, 0.002453153, 0.023572197, 0.04196399, 0.022900663, -0.0064926324, 0.0010270008, -0.011333841, -0.031986922, 0.024833035, -0.018528843, 0.0059752776, -0.020310462, 0.003511846, 0.0012488468, 0.015130061, 0.0014372873, -0.017405052, -0.007900797, 0.0066159754, 0.0012925307, 0.024942674, -0.0011426348, -0.006304192, 0.02806736, -0.009072554, 0.017035022, -0.0025148245, 0.014760032, 0.023215873, -0.005214663, 0.007174445, -0.6543203, 0.00011938135, 0.02327069, -0.040757973, -0.0003952968, 0.022407291, 0.00931924, -0.010662307, -0.017240595, 0.029026693, 0.008435283, -0.013094903, 0.0035563866, -0.0032531687, -0.00036231973, -0.014924489, -0.0035067068, -0.013937746, 0.002631315, 0.02714914, -0.021557596, 0.018199928, -0.028971873, 0.0072977873, -0.00372427, 0.008106369, 0.0016231581, -0.02660095, -0.0015991748, 0.003081859, -0.021256091, -0.0034261914, 0.0024086125, 0.0050227963, 0.055942852, -0.008524364, -0.02044751, 0.008058402, 0.004406082, 0.031712826, -0.045033857, -0.0001949717, 0.020365281, -0.008784754, -0.0004055754, 0.014280366, 0.008394169, -0.0075102113, -0.0084421355, -0.013711617, 0.0033388236, 0.0031606616, 0.011121417, -0.004159396, 0.002489128, -0.01153256, 0.021187568, -0.008462693, 0.005800542, -0.016061986, -0.0039880867, -0.004460901, -0.0140199745, -0.0066844993, -0.016541652, -0.00092335854, -0.0031743662, 0.024764512, -0.00788024, -0.022722501, 0.005015944, 0.033384807, -0.017418757, -0.004632211, -0.0023949076, 0.020488624, -0.0016180188, -0.019680044, -0.029492656, 0.017665442, 0.009764645, -0.026093872, -0.03689323, 0.024147797, -0.006071211, -0.0019820516, -0.01693909, 0.013273065, -0.0052009583, 0.013560865, 0.012019079, 0.007962469, -0.022407291, -0.054490145, -0.008394169, 0.0019203803, 0.0024463006, 0.013677356, 0.025148245, -0.008270826, -0.0064241085, -0.012676908, 0.010977517, 0.008257121, 0.023654426, 0.015308223, 0.004245051, 0.017884718, 0.0024274567, -0.016500538, -0.01371847, -0.0064926324, 0.0075787352, -0.0130606415, -0.01095696, -0.02812218, 0.01906333, 0.016226443, -0.016212737, -0.008126926, 0.0113201365, 0.020406395, 0.011306431, 0.0024497267, 0.019501882, 0.003528977, 0.002024879, -0.0075444733, -0.030287532, 0.0043546893, 0.0033148401, -0.0029705078, 0.019762272, -0.024339663, 0.0129715605, -0.004889175, 0.007866535, -0.018775528, 0.0045739654, -0.0163909, -0.042978145, -0.0039641033, -0.0013422105, -0.012560417, -0.021077929, -0.019049624, -0.010285426, 0.0076061445, 0.0035186985, -0.0060232445, 0.0025285291, -0.016747223, -0.021091634, 0.014828556, 0.020981995, -0.0034416092, 0.015116356, -0.039826047, -0.0022218851, -0.03374113, 0.018515138, 0.025203064, -0.009415174, -0.004210789, -0.01110086, -0.010573226, 0.0019769124, 0.01780249, -0.03549534, -0.019501882, 0.009983921, -0.02691616, -0.009682417, 0.015280813, 0.01127217, 0.010484145, -0.016692404, -0.00672904, -0.00019582825, -0.036673952, 0.005194106, 0.008709379, -0.007270378, -0.018103994, 0.014801146, -0.004693882, 0.026409082, 0.009730384, 0.028396273, 0.0050810417, -0.0037448271, 0.011813507, -0.013293622, 0.023298101, -0.0007910219, 0.022722501, 0.0061465874, 0.01963893, 0.0048686177, 0.013643093, -0.012676908, -0.007229264, 0.005992409, -0.02217431, 0.012896184, -0.03741401, -0.0040874463, -0.02864296, 0.0077637495, 0.0039675296, 0.0034792973, -0.005663494, -0.018446615, -0.005684051, 0.0122794695, 0.034316733, 0.0031949235, 0.031740237, -0.0010749676, 0.022092082, -0.015527499, -0.010408769, 0.023709243, 0.0010132961, -0.0017884718, 0.0021430827, 0.0037996462, 0.016747223, 0.009093111, -0.022448406, 0.00819545, 0.000523779, -0.017377643, 0.014705213, 0.007852831, -0.0021550744, 0.02568273, -0.0007199285, 0.047528125, 0.006852383, 0.016856862, 0.009250716, 0.014403708, -0.0016582767, 0.008462693, 0.017350232, 0.027944017, -0.021242386, -0.01636349, -0.0069448897, -0.0345086, 0.019762272, -0.0114092175, 0.0038955796, 0.014047384, 0.015390451, 0.00259534, -0.012519303, 0.0020625673, 0.036098354, -0.015993461, -0.0151574705, -0.006016392, -0.01124476, -0.0004466897, -0.018474024, -0.0044334917, -0.027971426, -0.012560417, 0.043170013, -0.0003475443, -0.0139514515, 0.00759244, -0.02505231, 0.008901245, -0.0031212603, 0.0030578757, -0.0014895367, 0.00874364, 0.014088498, -0.019748567, -0.02154389, 0.017720262, 0.0123685505, 0.00095933356, -0.007866535, -0.019625224, 0.010662307, -0.009990774, -0.009627597, 0.0023555066, 0.010607488, 0.01722689, -0.007249821, -0.0027152565, 0.015719365, 0.01225206, -0.005663494, 0.0009456288, 0.009702974, -0.012245208, 0.027478054, 0.00013940316, -0.022818435, 0.032480292, 0.009812612, -0.016432013, -0.0047384226, 0.0064857802, -0.022064673, 0.025477158, -0.0040874463, -0.0012616949, 0.006787285, 0.017953243, -0.008805311, -0.025490863, -0.0042142156, 0.03204174, -0.011374955, 0.016459424, -0.02484674, -0.011765541, 0.026381673, 0.012622089, 0.017665442, -0.014760032, 0.00032313267, -0.011374955, 0.013985713, -0.010977517, -0.02927338, 0.01463669, -0.00908626, -0.027848084, 0.010086707, 0.00027216808, -0.015815299, 0.009798907, 0.009257569, -0.010915846, -0.025902007, -0.027779559, -0.009367207, 0.01403368, 0.0139514515, 0.0033593806, 0.004368394, -0.006047228, -0.024065567, 0.008791607, 0.034152277, -0.0037859415, -0.04363597, -0.0060575064, 0.013355294, -0.008558626, 0.018515138, 0.0020026087, 0.023448853, 0.01900851, -0.00059187453, 0.03094536, 0.022050967, 0.009113668, 0.02795772, 0.017761376, 0.008222859, 0.0121903885, -0.012347993, 0.006012966, 0.030643856, -0.0068215467, -0.02743694, 0.0033388236, -0.017007614, -0.019926729, 0.0065817134, -5.455139e-05, 0.0010689717, 0.014170728, -0.0027323875, -0.007325197, -0.014787442, -0.004080594, -0.02869778, 0.016486833, -0.025929416, -0.008846426, -0.0019923302, 0.0004374818, 0.0029465246, -0.007393721, -0.0051975325, -0.0047658323, -0.009435731, -0.037112504, -0.0012154414, -0.011080302, 0.021256091, 0.025011197, -0.030753493, -0.015979756, -0.01095696, -0.014732623, -0.014458528, -0.005457923, -0.030506808, -0.007935059, 0.030835722, 0.0014595576, -0.0068489565, -0.018035471, -0.0069654468, 0.013663651, -0.0007045106, 0.009689269, -0.007140183, -0.0062425206, 0.009175341, 0.0031007032, 0.015445271, 0.014883375, -0.018816642, -0.011039188, -0.024778215, -0.0011066598, -0.016459424, 0.018528843, -0.0059478683, -0.005759428, 0.007660964, -0.0017953243, -0.021804282, 0.015637137, 0.00533458, 0.0079076495, 0.0022424422, 0.023517378, -0.009840022, -0.0032000628, 0.022146901, 0.016267557, -0.012293175, -0.006496059, -0.022722501, 0.019830795, -0.009079407, -0.008675116, 0.031274274, 0.010148379, -0.006047228, -0.038071837, 0.010189493, -0.014458528, 0.047473308, -0.01774767, -0.02079013, -0.022160605, -0.017555805, -0.0031812186, -0.011854622, -0.010340245, -0.005512742, -0.0026039055, 0.0327818, 0.002972221, -0.020899767, -0.0072429683, -0.028917056, 0.0041491175, 0.020008957, 0.007290935, 0.020008957, -0.0024942672, 0.007859683, 0.011436626, -0.032507703, -0.010648603, -0.0362354, -0.014787442, 0.0034159129, 0.011306431, -0.016061986, 0.03456342, -0.016815746, -0.0042553297, 0.029629702, 0.0115873795, 0.020173414, 0.0058108205, -0.0057765585, -0.026559835, 0.004121708, -0.0017533534, 0.0040908726, 0.030726084, -0.029903797, 0.042895917, 0.01374588, -0.018857757, 0.0037105652, 0.004570539, -0.018871462, -0.028368864, 0.020091185, -0.017679147, -0.00035760872, -0.03801702, -0.0029259673, 0.009812612, -0.0027615102, 0.013094903, 0.020241939, 0.03683841, -0.0037653842, -0.0033799377, -0.006071211, 0.026628358, -0.022188015, -0.007647259, -0.037852563, -0.007256673, 0.02858814, 0.010347098, 0.013471784, -0.028026246, 0.010490998, -0.008490102, 0.018707005, 0.0002224883, 0.0022904088, -0.017377643, -0.01064175, -0.012978413, -0.029684521, -0.030451989, -0.016993908, -0.01340326, 0.015883824, -0.013485489, 0.030068256, -0.017583214, -0.018460318, 0.009545369, 0.02680652, 0.021667235, -0.0042313463, 0.020611968, 0.018569957, 0.008120073, 0.011895737, 0.0015452122, 0.004385525, 0.0022989744, 0.026683178, 0.01239596, 0.005152992, -0.018871462, 0.015582318, 0.0016060271, -0.029410426, -0.03143873, 0.012704317, -0.01863848, 0.009278126, -0.0057936898, -0.011957408, -0.013122313, 0.0018758398, -0.027190255, 0.0149793085, 0.004118282, 0.005951294, -0.014458528, 0.040483877, 0.016473128, 0.03280921, -0.009785202, 0.008846426, 0.004995387, 0.019104443, 0.0017970373, 0.018227337, -0.00533458, 0.02982157, -0.008853278, 0.0037927937, 0.01602087, -0.0004955129, -0.012437074, -0.0052934657, -0.018090291, 0.029410426, -0.04094984, 8.05155e-05, -0.0018295862, -0.0043067224, 0.0033234055, -0.0031760794, -0.0137732895, -0.01486967, -0.0029670817, 0.009332946, 0.016610175, -0.00035311186, -0.0186933, 0.011882031, -0.0006925189, -0.0122794695, -0.021639824, -0.014814851, -0.014993013, -0.043032963, -0.002934533, 0.0029259673, -0.004025775, 0.019392243, -0.0025525126, -0.01426666, 0.019830795, 0.03439896, -0.019885615, 0.026614655, 0.015966052, -0.0064172563, -0.006180849, 0.015691957, 0.008099516, 0.0015023849, 0.031411324, -0.018515138, -0.008928655, -0.0032257591, -0.0224347, -0.015034128, 0.0038887272, -0.0063144704, -0.031137226, -0.007469097, 0.03261734, -0.028560732, -0.014650394, 0.000939633, -0.01328677, 0.018309567, 0.014143318, -0.030835722, 0.007948764, -0.014184432, -0.008113221, 0.024654873, -0.04018237, 0.018186223, -0.009168488, 0.019118147, -0.0095590735, -0.02285955, 0.0026398804, -0.0025747828, 0.0019649207, 0.007914502, 0.0090451455, -0.032480292, 0.02079013, -0.0026296019, 0.00049165846, 0.0033216926, -0.01785731, -0.00591018, -0.0089560645, -0.022681387, -0.020529738, -0.014951899, -0.013547161, 0.042457364, 0.021256091, -0.010908993, -0.019131852, -0.011772393, -0.058053385, 0.013752732, 0.007195002, -0.0025233899, 0.0057148873, 0.012533008, -0.00090879726, 0.019734861, -0.0123685505, 0.012793398, 0.017117253, 0.0026998387, -0.0016762641, -0.0058793444, -0.012032784, 0.005875918, 0.0013516325, -0.009182192, 0.011882031, 0.013622536, -0.003222333, 0.020461215, 0.020899767, 0.010168936, 0.003528977, 0.028505912, 0.014924489, 0.007284083, -0.0025542257, -0.01693909, 0.007023692, 0.008078959, 0.007078511, -0.022338767, -0.012628941, 0.0066639422, 0.021776872, 0.015116356, -0.020036368, 0.014211842, -0.005903328, -0.0003447605, 0.0362354, 0.010730831, -0.01253986, 0.03152096, 0.018569957, 0.0113201365, -0.01604828, 0.008914949, -0.02764251, -0.0013995991, 0.010319688, -0.018268453, -0.010305984, 0.0032548818, 0.016500538, -0.024545236, -0.007482802, 0.0023161054, -0.0046116537, -0.0010355663, 0.037441418, 0.028752597, -0.012704317, 0.0025936267, -0.012224651, -0.005492185, -0.01434889, 0.009572779, -0.00555043, -0.013252508, 0.031877283, 0.006067785, -0.0069757258, 0.009339797, 0.005211237, 0.029574884, 0.005519594, 0.029108921, 0.2128076, -0.026272034, 0.013512898, 0.024983788, -0.0022047542, -0.0092918305, 0.023805177, 0.0010535539, -0.022530634, 0.022023559, 0.0011854622, 0.0021893363, 0.009682417, -0.0050056656, 0.015774185, -0.023627015, -0.017720262, -0.007523916, -0.008832721, 0.0015452122, 0.018926281, -1.1349259e-05, -0.002006035, -0.015349337, 0.05248925, -0.017706556, -0.027478054, -0.016322376, 0.007147035, 0.026696883, -0.024367074, -0.0050913203, 0.019570405, 0.012992118, -0.0064241085, 0.0063110446, 0.009353503, -0.0076198494, 0.008325645, 0.026642064, 0.04678807, -0.0058039683, 0.012025932, -0.017816195, -0.016377194, 0.017117253, -0.020310462, -0.014883375, -0.0230103, 0.004892601, -0.006249373, -0.0013122313, 0.027272483, 0.025066016, -0.019145558, 0.0043958034, -0.023764063, 0.015966052, 0.03094536, -0.008058402, -0.0040942985, 0.0018946838, 0.00032934264, -0.0076541114, -0.010806208, 0.016760929, 0.00934665, 0.009675564, 0.021955034, -0.027478054, 0.004296444, -0.014622984, -0.0051632705, -0.00076875166, -0.0090451455, -0.033467036, 0.02505231, 0.01486967, 0.026121283, 0.03322035, -0.013663651, -0.0064857802, -0.004885749, -0.023147348, -0.026724292, -0.06534432, 0.009860579, 0.0070990683, -0.01486967, -0.024764512, 0.0016796903, -0.008401021, 0.006732466, -0.0017816195, 0.0044026556, 0.0009113669, -0.003261734, -0.009600188, -0.008284531, 0.00048994535, -0.0212835, 0.05029649, 0.026833931, 0.016856862, 0.03908599, 0.0010698283, -0.015637137, 0.0029807864, 0.008017288, -0.0192689, 0.0008672547, -0.021379434, 0.009175341, -0.004114856, -0.019172966, 0.011902588, 0.013362146, -0.023174759, 0.003994939, 0.005012518, -0.031082409, -0.0215713, 0.0010526973, 0.03267216, 0.012663203, 0.0013756158, -0.03330258, -0.0035563866, -0.016541652, -0.013896632, 0.0056600682, -0.016459424, 0.0034244782, 0.010819912, -0.0057834107, -0.001880979, 0.009442584, -0.009065703, -0.014993013, -0.0049576988, -0.00060172484, 0.009483698, 0.019049624, -0.02565532, 0.008873835, -0.030314941, -0.0031298257, 0.0130606415, -0.0056121014, -0.02010489, -0.01848773, -0.019460768, -0.0008942359, -0.006674221, -0.003169227, -0.013985713, -0.024147797, -0.034015227, 0.015664546, 0.030040845, -0.049309745, 0.006304192, 0.011210498, -0.019035919, 0.0005126439, -0.024380777, -0.17596918, -0.0014210128, 0.008640855, -0.03459083, 0.015554909, 0.007921354, 0.030589037, 0.009977069, -0.03311071, 0.008243416, 0.0011186515, 0.0056532156, -0.028204408, 0.0070648063, -0.0207079, 0.011929998, -0.027012093, 0.0013559152, 0.020831244, 0.005485332, 0.025038606, -0.041772123, 0.016829452, -0.00017045302, -0.0026296019, -0.015678251, -0.009442584, 0.026148692, 0.014691508, -0.021077929, 0.012053341, -0.015239699, 0.01748728, -0.019652633, 0.006396699, 0.01049785, 0.009442584, 0.0029054102, -0.017185776, 0.004190232, 0.008702526, 0.015609727, 0.0081748925, -0.0031658008, -0.00658514, 0.0391134, 0.007715783, -0.0037037127, -0.0060917684, 0.0010749676, 0.013321032, -0.014143318, -0.00112893, 0.011614788, 0.026272034, 0.00033919295, 0.00569433, 0.007236116, -0.006430961, 0.00068181206, -0.006468649, -0.026066463, 0.016733518, 0.0010843895, -0.01848773, -0.013540308, -0.023106234, 0.009483698, -0.008551774, 0.014417413, 0.0037825152, -0.010847322, -0.014239252, -0.0011906015, -0.0059547205, 0.024600053, 0.0055709872, -0.001915241, -0.006715335, -0.016061986, -0.017528394, 0.015349337, -0.0075102113, 0.00046681857, 0.0071538873, 0.00075676, 0.026148692, 0.0072977873, 0.00014272229, 0.00083684723, 0.019940434, -0.015554909, 0.006677647, -0.009723531, 0.027299892, 0.0016702682, 0.0004072885, 0.0002340517, 0.008551774, -0.010223755, -0.0023794898, -0.0088669835, -0.0056669204, -0.009113668, 0.03094536, 0.012937298, -0.020406395, 0.012046489, 0.034070045, -0.021352025, -0.007434835, -0.008154335, 0.004248477, -0.0033371104, 0.015116356, 0.038373344, 0.011580527, -0.01602087, 0.026463902, 0.012807103, 0.05246184, 0.0010167223, -0.01843291, 0.01035395, 0.016884271, -0.035988715, -0.12904407, -0.015500089, -0.0055983965, -0.006914054, -0.01426666, 0.041141704, -0.00593759, -0.00032720127, -0.027834378, 0.013499194, -0.008435283, -0.031274274, -0.00302704, -0.011621641, -0.009298683, -0.0052009583, -0.0040154965, -0.010484145, -0.022886958, 0.03634504, -0.011977965, -0.0076541114, 0.009230159, -0.015349337, -0.005029649, 0.03097277, -0.027916607, 0.0034124865, 0.005588118, -0.0008291383, 0.016432013, -0.014842261, 0.021160157, -0.040127553, -0.0022287373, -0.0025542257, -0.005036501, -0.00805155, 0.012704317, -0.0169665, -0.0060849157, -0.004097725, 0.0054544965, -0.0045671132, 0.0043820986, -0.023448853, -0.010305984, 0.028341455, 0.007030545, -0.019118147, -0.055586528, -0.017651739, -0.028094769, -0.0029088364, 0.019693747, -0.009833169, 0.0030047698, -0.025833482, -0.016925385, 0.00080301357, -0.047610354, -0.009744088, -0.02280473, -0.0031109818, -0.010443031, 0.0048240772, -0.035961304, -0.026655769, 0.007126478, -0.013081199, -0.016733518, 0.024545236, -0.004358115, 0.02369554, -0.0460206, -0.017816195, -0.0006090055, -0.010655455, 0.011601084, -0.012786546, -0.01400627, -0.018994804, 0.007921354, -0.022283949, 0.0030390318, 0.0074553923, -0.014842261, -0.0010038741, -0.0057217395, -0.02505231, 0.00083684723, 0.01460928, -0.006824973, -0.016706109, -0.0027289614, 0.0043786727, -0.010799355, -0.0040154965, 0.022023559, 0.012574122, -0.011402365, -0.010299131, -0.015952347, 0.021927625, 0.0031760794, -0.033933, -0.013492341, -0.025600502, 0.0022458683, 0.010943255, -0.023859996, 0.0011734705, -0.008421578, 0.0075650304, -0.010730831, -0.007201854, -0.0204201, -0.017446166, 0.009401469, -0.0032977092, 0.015966052, 0.025518272, -0.013588275, -0.0024240303, 0.0019101016, 0.010634898, -0.008873835, 0.02602535, -0.018391795, 0.04020978, -0.0136842085, -0.017816195, 0.020954587, -0.025381226, 0.00093278056, 0.01987191, 0.002398334, 0.01832327, 0.012868775, 0.04158026, 0.04078538, 0.035933893, -0.030095665, -0.013643093, 0.0046973084, -0.0114982985, 0.020611968, 0.021735758, -0.04092243, -0.023421444, 0.016719814, 0.003044171, 0.031356502, -0.006927759, -0.0067941374, -0.019474471, -0.015705662, -0.004570539, 0.007839126, 0.0054339394, 0.023339216, -0.013972009, 0.047747403, -0.016199032, -0.0055846917, -0.012649498, -0.026272034, -0.004423213, -0.0085312165, 0.025806073, -0.009504255, -0.00189811, -0.009257569, 0.00802414, 0.0040189223, 0.005752575, 0.019693747, -0.003994939, 0.00903144, 0.006427535, -0.001496389, 0.030479398, -0.0120122265, 0.004549982, -0.024147797, -0.008243416, 0.03686582, 0.011333841, -0.0035049936, -0.0017319397, -0.015431566, 0.003167514, 0.014622984, 0.0038578915, 0.0014458527, 0.015486385, 0.009901693, 0.009065703, -0.013005822, 0.043279648, 0.013321032, 0.016925385, 0.023106234, -0.010676012, -0.013101756, -0.030287532, -0.005296892, 0.026463902, -0.015527499, -0.025559388, 0.0075102113, 0.030287532, -0.0007293505, 0.0024565791, 0.0046561942, 0.002790633, -0.023599606, 0.030534217, -0.029629702, -0.014513346, -0.026066463, 0.028889645, 0.018460318, 0.014499642, 0.015541204, 0.002848878, 0.016240146, -0.00034583118, 0.013807551, -0.020502329, 0.014718918, 0.0039127106, 0.00083984516, 0.0054750536, 0.015171175, -0.002024879, -0.0007190719, 0.001683973, -0.0068078423, 0.028752597, -0.0025353816, 0.07559548, 0.021228682, -0.0027358138, 0.009353503, -0.025477158, -0.0026741424, 0.0046561942, 0.022133196, -0.02248952, -0.025627911, 0.014568166, 0.010203198, 0.023161054, -0.049830526, 0.008846426, 0.012861922, -0.022160605, 0.031740237, -0.014760032, -0.008962916, 0.027450645, 0.03258993, 0.009442584, 0.017980652, -0.027711036, -0.018871462, 0.0051324344, -0.0022253112, -0.014677804, -0.005930737, -0.01780249, 0.010223755, -0.03798961, -0.0036934342, -0.0051461393, -3.1719035e-05, 0.00098503, -0.0040942985, 0.011169383, 0.007681521, 0.013588275, 0.02573755, 0.0004942281, -0.019693747, -0.01271117, -0.00067581626, 0.035879076, -0.013307327, -0.031137226], index=2, object='embedding'), Embedding(embedding=[-0.014035202, 0.0072285635, -0.0036639199, -0.014246165, -0.016839761, 0.016666027, -0.014680498, -0.019420948, -0.044326916, 0.0015775642, 0.008637047, 0.00044674383, 0.0035491316, 0.0052740593, -0.025265846, -0.008078618, 0.014481946, -0.02180358, 0.024496453, -0.013352676, -0.0013014517, -0.0031163485, -0.008283375, 0.005503636, -0.025464399, -0.008066208, -0.003986568, 0.0011858877, 0.005233728, -0.032562662, 0.009449873, -0.015971093, -0.015660854, -0.023863567, -0.027822213, 0.018006258, -0.005553274, -0.026854267, 0.031197611, -0.006577062, 0.010088965, 0.00064296986, -0.006378509, -0.026357885, -0.035441678, 0.028417872, -0.0013053296, 0.0003544478, -0.020885274, 0.036335167, 0.011205824, -0.009468487, -0.021406475, -0.03918936, -0.0027642276, 0.021878038, -0.01655434, 0.0048738513, -0.01938372, -0.0054912264, 0.007786993, -0.019507814, -0.0035491316, -0.009133429, 0.012105517, 0.000967945, 0.014208936, 0.028095223, 0.01008276, 0.013935925, 0.02212623, 0.03479638, 0.011906964, -0.01682735, -6.170843e-05, 0.009940051, 0.0021018677, 0.0069679627, -0.0054136664, -0.005243035, 0.018726012, -0.033257596, 0.0037383772, 0.012490213, 0.034101445, 0.003400217, -0.0054136664, 0.007569826, -0.016206874, -0.016901808, -0.0071727205, 0.015983501, -0.02042612, 0.012223408, -0.011441606, -0.0033505787, -0.01822963, 0.02185322, -0.013402315, -0.008438495, 0.01599591, -0.007867655, -0.016951447, -0.011975217, -0.01548712, -0.014519175, 0.002647888, -0.0021266867, -0.0033785002, -0.01118721, -0.0065894714, 0.022796344, 0.0030356864, -0.034101445, 0.014742547, 0.0046163527, -0.01599591, -0.014990738, -0.04668473, -0.0153754335, 0.02074877, 0.018651554, 0.018093126, -0.004808701, 0.026730172, 0.0072037443, -0.01062878, -0.039809838, 0.0046752985, 0.008959696, 0.003992773, 0.016219283, 0.020525398, 0.030750867, -0.012484008, 0.013700144, -0.020041425, 0.024769463, -0.0320911, -0.01105691, 0.0068686865, 0.0070610344, 0.011919374, 0.014519175, 0.009009334, 0.015437481, 0.018750831, -0.020475758, -0.008792167, -0.016120007, 0.027747756, 0.00042076132, 0.024148986, 0.009567764, -0.006992782, 0.0068438672, -0.016368197, 0.02043853, -0.01216136, -0.014395079, -0.020277206, 0.012657742, 0.028864615, -0.028988712, 0.022448877, 0.025613314, 0.0045729196, 0.0036515102, -0.019942148, 0.0012076044, -0.010939019, 0.02625861, -0.014233755, 0.008941081, -0.012285455, 0.012527442, 0.010572937, -0.008047594, -0.018639145, -0.017596744, -0.017212046, -0.00081747916, -0.0017109668, 0.03070123, -0.00414479, 0.009927641, 0.00980975, 0.00843229, 0.019371308, -0.011224438, 0.019482994, 0.014109659, 0.02263502, -0.0014317519, -0.6774622, -0.0348212, 0.0028138657, 0.018440593, 0.003967954, 0.020140702, 0.005441588, -0.00047039954, -0.03425036, -0.005050687, -0.019942148, 0.005953482, -0.01438267, -0.015090014, 0.0064777858, -0.014283393, -0.0013821138, -0.0068376623, 0.009983484, -0.0043619573, -0.00483352, -0.01739819, -0.014208936, 0.0010113784, 0.022138638, -0.0043247286, 0.0022662943, -0.0051096324, -0.010107579, 0.0068438672, -0.015958682, -0.013700144, 0.002118931, -0.018304087, 0.045170765, 0.0016054857, 0.01659157, 0.027449926, -0.0014131377, 0.02042612, -0.029584369, -0.0058293864, 0.0075015733, -0.0037787082, -0.02097214, -0.009449873, 0.013514001, -0.009611197, -0.000960189, 0.006490195, 0.015462301, -0.0033257597, -0.00606827, 0.017522287, 0.023801519, -0.005472612, 0.008028979, 0.0058293864, -0.013873878, -0.021480933, -0.019396128, -0.006121011, -0.019805644, 0.0079048835, -0.013886288, 0.004855237, -0.020587444, 0.0135636395, 0.029708466, 0.01548712, 0.00980975, 0.009722883, -0.015425072, 0.030750867, 0.01656675, 0.026481982, 0.019743595, -0.033927713, -0.0016411631, 0.007439526, 0.01846541, -0.0061179087, -0.01963191, -0.009623607, 0.013005209, -0.019098299, -0.020649493, -0.008717709, 0.021046598, -0.0011649466, 0.031917363, -0.0093443915, 0.0034994932, -0.022461288, 0.00011633954, 0.01576013, -0.029683646, 0.0060434514, 0.006009325, -0.021356838, -0.0036577152, -0.018366136, 0.027747756, 0.003034135, 0.01105691, -0.004697015, -0.025662951, -0.00828958, 0.009977279, -0.00925132, -0.0057921577, 0.004529486, -0.005385745, -0.0031706402, 0.04698256, -0.016901808, 0.03122243, 0.013787011, 0.0048273155, -0.0063660997, -0.008326809, 0.0005782075, 0.020053834, -0.014730137, -0.0036142818, 0.0062606186, 0.005351619, -0.0070548295, -0.00029104276, 0.00093071634, 0.007098263, -0.033381693, 0.008047594, -0.009611197, -0.008767348, -0.018726012, 0.021245152, -0.009418849, 0.024794282, 0.0020770486, -0.024905968, -0.0085687945, -0.012868704, 0.023751881, -0.00869289, -0.014568813, -0.012856294, -0.011547087, -0.0028759134, 0.0150155565, -0.009766317, 0.0073650684, 0.0026292736, 0.012384731, 0.010920404, -0.0042440663, -0.004541896, -0.0015915249, 0.0023035228, -0.004399186, 0.001991733, 0.021518162, -0.016790122, -0.006009325, 0.018105535, -0.01242196, -0.007104468, 0.021083826, 0.0040020803, -0.025241027, -0.02019034, -0.013402315, 0.0014077084, -0.021716714, -0.015809767, 0.010337156, -0.003744582, 0.0020336153, -0.004697015, -0.011776663, 0.013849059, 0.013762192, -0.014233755, 0.016169645, 0.031867728, -0.0027223453, 0.008606023, 0.035516135, -0.024409587, 0.013178943, -0.010430227, 0.005776646, -0.029485093, 0.0056215264, 0.03174363, 0.023875976, 0.014109659, -0.008872828, 0.0053764377, 0.027995946, 0.006384714, 0.0063009495, 0.025737409, -0.011677387, 0.01256467, -0.0008291131, 0.0054695094, -0.01880047, 0.029112807, 0.017236866, 0.0022089, -0.016442655, -0.014444717, 0.009710474, 0.006170649, -0.0024896662, -0.001712518, 0.007191335, 0.0012867153, 0.009636017, 0.015313386, 0.016095188, 0.014754956, -0.014060021, 0.0063660997, 0.008314399, 0.027971128, 0.023938024, -0.0038066297, -0.00077133114, -0.016182054, -0.0014914729, -0.0027223453, -0.0016023832, 0.013973154, 0.014159298, 0.01963191, -0.03228965, 0.019818053, -0.00061349716, 0.00052314013, -0.0051344517, 0.044525467, -0.012694971, -0.024111757, 0.020388892, 0.015573986, -0.020562626, 0.0006390919, 0.02486874, -0.0032233808, 0.021815991, -0.014345441, 0.0042192475, 0.008364038, -0.0016303047, -0.0072782016, 0.010306132, 0.029609188, 0.028566785, 0.0042037354, 0.024124166, 0.017199637, -0.019942148, 0.008078618, -0.00028173559, 0.036881186, -0.014444717, -0.016765304, 0.0029022838, -0.02376429, -0.011559497, 0.0059255604, -0.02131961, -0.00089659006, -0.00038741066, 0.020202748, -0.018328907, -0.007135492, -0.0070051914, -0.010740466, -0.031420983, 0.023416823, 0.02102178, -0.005649448, -0.0037973225, -0.0314458, 0.027648479, -0.0029069374, 0.01521411, 0.0066018812, 0.030527495, -0.00855018, 0.014208936, -0.014965918, 0.0045263837, 0.0041727116, -0.012558466, -0.019681547, -0.017844934, 0.007985546, 0.0062699257, -0.012471599, -0.020500578, 0.025389941, -0.01077149, 0.0027502668, -0.0025982498, 0.008246146, -0.008767348, -0.009598788, 0.0077745835, -0.0069617582, -0.0018815981, 0.017472647, -0.008643252, -0.0109142, -0.014630861, 0.031098334, 0.016157236, 0.013067257, -0.007898679, -0.036384802, -0.007762174, 0.109998256, 2.8963697e-05, 0.01022547, 0.005252342, -0.031396165, -0.00022938279, -0.011925578, -0.021468524, 0.016765304, -0.011540882, -0.015226519, 0.003186152, 0.013762192, 0.017162409, 0.0021964905, 0.0007814139, 0.011367149, 0.03268676, 0.0010067248, -0.010560527, -0.03174363, -0.0058200792, -0.0039989776, 0.0022507822, 0.01173323, 0.020624673, 0.038618524, 0.002056883, 0.007495369, 0.000552225, -0.002239924, 0.010467456, 0.025241027, -0.0072719967, 0.00050258683, 0.02846751, -0.018155172, 0.010219265, 0.008599819, -0.0034219336, 0.026581258, 0.01548712, 0.0019451971, -0.0042843972, 0.025638131, -0.014965918, -0.009449873, 0.02762366, 0.012943162, -0.009158248, 0.026854267, -0.017174818, -0.023801519, -0.027003182, -0.0070858537, 0.014481946, -0.028541967, 0.009114815, -0.041298985, -0.010138603, 0.0032419951, -0.020078653, 0.018589508, -0.020575035, -0.005655653, -0.03509421, -0.012471599, 0.00079343567, -0.023069356, -0.01518929, 0.027052822, -0.016268922, -0.017112771, 0.027698118, 0.015573986, 0.0011315959, 0.007644283, -0.0056215264, -0.01687699, 0.026382705, 0.0031272068, -0.024558501, 0.015338205, 0.0026913213, -0.017609153, 0.021356838, 0.011801483, -0.02155539, 0.019644318, 0.014606041, 0.016765304, -0.009753907, 0.014184116, -0.012217203, -0.008233737, -0.0049576154, 0.008084822, 0.003009316, -0.029683646, 0.019135527, 0.0022833573, -0.044053905, 0.01435785, -0.00089659006, 0.011869735, 0.00075969717, 0.014345441, 0.05420492, -0.02515416, -0.00093536987, 0.011776663, -0.023540918, 0.02816968, 0.010970043, -0.018130355, 0.03889153, 0.029931836, 0.0027673298, -0.002320586, -0.0067073624, -0.0088045765, -0.021406475, 0.010920404, 0.010740466, -0.00066468655, -0.012297865, 0.008177894, -0.031048696, -0.0024788077, -0.0010206855, 0.012694971, 0.014817004, -0.0130796665, -0.02928654, -0.019147936, -0.013389905, 0.012223408, 0.018961793, -0.0050041513, -0.0009322675, -0.02677981, -0.007569826, 0.013290629, 0.0018660862, -0.020537807, -0.019979376, -0.020512987, 0.00026680535, -0.019396128, 0.027449926, -0.020637084, 0.035689868, -0.0018148968, -0.0045574075, -0.017807705, 0.00559981, -0.012223408, 0.010256493, 0.011584315, 0.014469536, 0.017050723, 0.012930752, 0.010870767, 0.00967945, -0.02042612, -0.0015992809, 0.00029996212, 0.0026075568, -0.014568813, 0.031147972, 0.015921453, 0.024707416, 0.008972106, 0.03065159, -0.015102424, 0.020388892, 0.019259622, -0.00372907, -0.008270966, -0.020289615, -0.008779758, -0.018093126, -0.012353708, 0.0085563855, -0.0057735434, -0.0016737381, -0.00067360594, 0.004237862, 0.012905933, -0.0010067248, 0.004271988, -0.022734297, 0.024074528, -0.010374385, 0.028914254, -0.042664036, -0.013464362, -0.0049824347, 0.01629374, -0.0029565755, -0.00484593, 0.010734262, -0.0008213571, 0.0026122103, -0.0052771615, 0.0028293775, -0.019904919, -0.04194428, 0.024111757, -0.0071416963, 0.0112802815, 0.0062947446, -0.03846961, -0.0012626718, -0.0070051914, 0.013178943, -0.019929739, 0.017609153, -0.019693958, -0.027499564, -0.0007364293, 0.013948335, 0.013576048, 0.01131751, 0.03710456, 0.039636105, -0.00010858357, -0.009034153, 0.0068624816, -0.010219265, -0.020587444, 0.009921436, 0.012645332, -0.031371344, -0.008122051, 0.0022554358, -0.0030729151, 0.004864544, -0.015896635, -0.0018117945, 0.011962807, 0.011509858, -0.012192383, 0.021245152, -0.026879087, 0.014519175, -0.00035095762, -0.018887337, -0.016504703, -0.00939403, -0.019073479, 0.02484392, -0.0133278575, 0.010517094, 0.014270984, -0.006201673, -0.0018567791, -0.0320911, -0.011689797, -0.0064529665, -0.0068128435, 0.018539868, -0.0022306168, 0.010684623, 0.016268922, 0.0021964905, -0.016070368, -0.0068438672, 0.010622576, 0.016244102, -0.0038283465, -0.0019157244, -0.0031458212, 0.017869754, 0.02623379, 0.013178943, -0.011348534, -0.015636034, -0.008506747, -0.016082779, 0.02541476, -0.004892465, -0.0027068332, -0.014320621, -0.0011184107, -0.034448914, -0.03375398, 0.015933864, 0.024446815, -0.0076691024, -0.008941081, -0.009722883, -2.4019268e-05, 0.00019225109, 0.003192357, 0.002041371, 0.024657778, 0.024421996, -0.008661866, -0.001631856, -0.005441588, 0.02125756, -0.023640195, 0.016008321, 0.023032127, -0.011857326, -0.0095243305, -0.003490186, -0.013650506, -0.00912102, 0.020525398, 0.015524348, 0.001864535, 0.006058963, -0.009859388, 0.009195477, 0.015884224, -0.017137589, -0.020599855, 0.034275178, -0.0079048835, -0.0035894625, 0.019420948, 0.023801519, 0.010535709, -0.014159298, -0.016517112, 0.018378545, 0.0060217343, -0.03539204, -0.002241475, 0.017509876, -0.014879052, -0.0076753073, -0.010821128, 0.01407243, 0.005506738, 0.03323278, -0.020326843, 0.0077187405, -0.0011982973, 0.01655434, 0.01852746, 0.030055933, -0.003465367, -0.006111704, -0.024781873, -0.029931836, -0.04204356, -0.022448877, -0.020351663, 0.012694971, 0.003977261, -0.005466407, -0.015822178, 0.0016876989, -0.005159271, 0.0034684695, -0.024359949, 0.014680498, 0.014941099, -0.014742547, 0.00031605575, 0.0019343387, -0.005208909, 0.029534731, -0.0057021887, 0.005087916, 0.009009334, -0.0126639465, 0.001616344, 0.0046194554, -0.013774602, -0.0038531655, 0.0063909185, -0.010020712, -0.003415729, 0.024086937, -0.016430246, 0.011274077, 0.0016861478, -0.0064963996, -0.008767348, 0.023975253, 0.0021825298, -0.012670152, -0.014519175, 0.0061830585, 0.009654631, -0.009499511, -0.005903844, 0.010157217, -0.011962807, 0.0150155565, -0.0071727205, 0.00014862376, 0.01547471, -0.0058045676, 0.02153057, -0.00095475983, -0.021121055, -0.007706331, 0.009425054, -0.012012445, 0.009102406, 0.014804594, -0.023689833, -0.0022538847, 0.024124166, -0.02211382, -0.03921418, -0.0050786086, -0.0069493484, -0.041596815, 0.008649457, -0.007073444, -0.022647431, -0.016169645, 0.022175867, 0.014742547, -0.035243124, -0.036062155, -0.01491628, -0.0018800469, 0.007898679, -0.020066245, 0.018936975, -0.017758068, 0.0032016642, 0.025191389, -0.01960709, -0.016541932, 0.005972096, 0.0040982543, 0.0075015733, 0.040653687, 0.2259531, -0.023751881, -0.0020134496, 0.02457091, 0.02903835, 0.03318314, 0.0030605055, -0.0010330951, 0.007644283, 0.029212082, -0.010318542, -0.014246165, -0.016132416, 0.0017699122, -0.0016892501, -0.021369247, -0.041323803, -0.0009322675, -0.007402297, -0.01991733, -0.010591552, 0.012794247, -0.0066949525, -0.005751827, 0.019272033, 0.021431295, -0.020860454, 0.0088356, 0.025514036, -0.0029782923, -0.01934649, 0.011447811, 0.0331335, 0.016256511, 0.008798372, -0.008221327, 0.025042474, -0.010448841, 0.031619534, 0.00912102, 0.009363006, 0.004026899, 0.0028790159, -0.019545043, 0.006515014, 0.010585347, 0.0017171716, -0.0130796665, 0.010597756, 0.016169645, -0.033803616, -0.027598841, -0.008177894, 0.0204013, 0.018577097, -0.0033164525, 0.009282344, -0.011633954, -0.034424093, 0.008581204, 0.022845984, 0.022002134, -0.010355771, -0.00023287298, -0.018353725, 0.007948318, -0.01903625, -0.0027440619, 0.008208917, -0.0193589, 0.0116960015, -0.0061954684, 0.005618424, 0.013588458, -0.05192156, -0.021642257, -0.02150575, 0.0073898877, 0.030800505, 0.01270738, -0.012533646, 0.01661639, -0.0016768406, -0.0055656834, -0.01464327, -0.009555354, 0.020004196, 0.011609135, 0.0045698173, 0.004271988, 0.01435785, -0.019234804, -0.0070672394, -0.0031008364, 0.017633973, 0.01768361, 0.001776117, 0.020736359, 0.0057952604, -0.004123073, -0.006409533, 0.0049017724, -0.0014015037, -0.025538856, -0.017969029, -0.019284442, 0.026432343, 0.028839797, 0.005671165, -0.0032606095, -0.000498321, 0.0012099312, -0.00051383296, 0.007824222, 0.020872865, -0.0005979852, 0.0035398244, -0.00815928, 0.005897639, -0.008258556, 0.01742301, -0.021307198, 0.0019762209, 0.01798144, 0.02540235, 0.0005033624, -0.007594645, -0.0057828506, 0.022734297, -0.009698064, 0.0013464362, 0.025340304, 0.0358636, -0.024099348, -0.017609153, 0.017025903, 0.013377496, 0.015598806, -0.022734297, -0.0033505787, -0.004535691, -0.01008276, -0.006850072, -0.013303039, -0.021332018, -0.029907018, 0.012217203, -0.006114806, 0.0033629884, -0.025079703, -0.023032127, -0.013650506, -0.016120007, -0.0029674338, 0.046535816, 0.0072161537, -0.024421996, -0.010337156, 0.012415756, -0.014208936, -0.006186161, -0.025787046, 0.007129287, 0.026928725, -0.028219318, 0.010864561, -0.15655889, 0.014171707, -0.007818017, 0.005357824, -0.0010276659, 0.008091027, -0.010182036, -0.00469081, -0.011274077, 0.014494356, 0.017869754, -0.017212046, -0.009629811, -0.002135994, -0.008053798, -0.008593614, -0.0034498551, 0.023677424, 0.010591552, 0.004175814, 0.0017699122, -0.0059100487, 0.0013596214, 0.028120043, 0.020215157, 0.006688748, -0.04008285, 0.023590555, -0.013451953, 0.0034529576, -0.049663022, 0.0022352703, 0.001776117, 0.0054260762, 0.02210141, -0.003490186, 0.0042223497, 0.026060056, -0.012831476, 0.010299928, 0.03506939, 0.009834569, -0.0055501717, 0.0064777858, -0.03310868, 0.019495405, 0.021617437, -0.0072533824, -0.010306132, -0.031396165, 0.014258574, -0.03477156, 0.0020584343, -0.002903835, -0.0031566794, 0.018899746, 0.005497431, 0.009201682, 0.019284442, 0.0072533824, -0.033083864, 0.007538802, 0.0051809875, -0.00241676, -0.0095925825, -0.031321708, -0.020066245, 0.0015783397, -0.023019716, 0.02816968, 0.009685654, -0.034374457, 0.015139652, -0.025935961, 0.01326581, 0.0012037264, -0.040653687, 0.01630615, -0.00020475758, -0.045741603, 0.0038593702, 0.024198623, -0.022572974, 0.0072161537, -0.015288566, 0.011001067, 0.012322684, 0.014965918, 0.0109079955, -0.01160293, 0.011727026, -0.018068306, 0.00018275002, -0.022002134, 0.03181809, 0.001783873, 0.020326843, 0.0009105508, -0.010343361, -0.010250289, 0.007178925, -0.028939072, -0.01687699, 0.012390937, 0.016678436, 0.016541932, -0.015971093, 0.0005471836, 0.01821722, 0.0015178432, -0.0104922755, -0.011813892, 0.020637084, 0.020475758, -0.019135527, 0.009890412, -0.0072782016, -0.017919391, 0.0064529665, 0.0008725465, 0.030999059, 0.0055749905, 0.0073898877, 0.013849059, 0.015710492, 0.010684623, -0.07396092, -0.014419898, 0.008252352, 0.024421996, -0.014444717, 0.026357885, -0.0075325975, 0.04670955, -0.002016552, 0.051723007, 0.0040020803, -0.0072037443, -0.027052822, -0.003192357, 0.008382652, -0.0068686865, -0.017633973, -0.00049444305, -0.0021794273, 0.010659805, -0.03812214, -0.027747756, -0.0019312364, -0.006384714, -0.023838747, -0.0122358175, -0.04621317, 0.006986577, 0.013824239, -0.014494356, 0.010026917, -0.009673245, 0.012490213, -0.036136612, -0.0150155565, 0.018018669, -0.025712589, -0.0038593702, 0.011429196, -0.03645926, -0.0013728066, -0.0023485075, -0.026134513, -0.029857378, 0.007098263, 0.0037166604, 0.0026199664, 0.0020832533, -0.013935925, -0.027325831, -0.023354774, -0.0011478835, -0.026978364, -0.002345405, 0.022734297, -0.021493342, -0.004064128, 0.02983256, -0.021840809, 0.006136523, -0.0012556914, -0.004541896, -0.008258556, -0.0002268621, 0.017001085, 0.010715648, -0.019433357, 0.008593614, 0.023069356, 0.009623607, -0.003176845, 0.0064033284, -0.033878073, 0.015623624, -0.031396165, 0.0057952604, 0.0013976257, -0.015350615, 0.005162373, -0.0077249454, -0.012757018, -0.004185121, -0.019582272, -0.0015744618, 0.013141714, -0.0045977384, -0.0101882415, -0.0063071544, 0.017770477, -0.029087987, 0.010697033, 0.010299928, 0.009331983, -0.005568786, -0.018366136, -0.0043743667, -0.0048055984, -0.012732199, 0.023677424, -0.008066208, -0.030527495, -0.005311288, -0.049166642, 0.02849233, 0.0066204956, 0.0029829459, -0.010057941, -0.024421996, 0.0030387887, -0.034920476, 0.005559479, 0.0042937044, 0.012093107, 0.024446815, 0.0028216217, -0.0031147972, -0.010399204, -0.010175832, 0.022970079, -0.011019682, 0.0058480008, -0.0048180083, 0.009698064, 0.0008430739, 0.007346454, -0.0051313494, 0.008475724, 0.014308212, -0.030180028, 0.008792167, -0.023118993, -0.0148914615, 0.022672249, -0.03010557, -0.0004544998, 0.019507814, -0.009034153, -0.027276192, 0.00033505788, 0.014817004, -0.023329956, -0.02150575, -0.006127216, -0.027772576, 0.017894572, -0.00066778896, 0.0109762475, 0.016765304, -0.0304034, -0.007700126, 0.019793233, 0.019123118, -0.0027238964, 0.027673298, -0.025191389, 0.0006088436, -0.00040990298, -0.031073516, 0.003986568, -0.025092112, 0.010721852, -0.008488133, 0.03730311, 0.0113361245, 0.016380606, 0.0042564757, 0.0029146932, 0.005143759, -0.00019448093, 0.008829395, -0.0021158285, -0.04591534, -0.017249275, 0.00046186798, 0.016169645, 0.022771526, 0.02680463, 0.0012944713, 0.0006084558, 0.0076877167, -0.023925614, 0.04139826, 0.025638131, 0.01353882, -0.014333031, -0.003977261, 0.037675396, 0.018080715, -0.025861504, 7.3439332e-06, 0.013476772, 0.0013921965, -0.06105499, 0.0015341308, -0.01744783, 0.01490387, -0.004945206, 0.0066204956, 0.014345441, -0.026903907, 0.0030310329, 0.019793233, 0.00456051, -0.005395052, -0.023528509, -0.0024462326, -0.028591605, -0.004334036, -0.019048661, -0.015276157, 0.014122069, 0.01881288, -0.0075574163, 0.015263747, 0.01634338, 0.01963191, 0.0025579187, 0.018080715, 0.0072161537, 0.005298878, -0.03174363, 0.040653687, -0.012099312, 0.007768379, 0.010411614, -0.017621562, 0.022597792, 0.013054848, -0.0047094245, -0.013613277, 0.0035894625, -0.009685654, 0.0051406566, -0.0103123365, -0.013166534, -0.0116960015, -0.016393017, -0.0031334115, -0.01521411, 0.023875976, 0.0027006285, 0.057083935, -0.012819066, -0.026010418, 0.006273028, 0.010603962, 0.0005351619, 0.013873878, 0.0061551374, -0.01435785, -0.014122069, 0.01852746, -0.0042223497, 0.009741497, -0.0010928161, 0.0028510944, -0.019048661, 0.008153075, 0.003176845, -0.019011432, -0.00053671305, 0.028814977, 0.0047187316, 0.012868704, 0.011553292, -0.025613314, -0.014618451, 0.015958682, -0.0044146976, 0.0028588502, -0.031396165, 0.007929703, 0.0054136664, -0.019197576, 0.0017590538, 0.011838712, -0.010572937, -0.022821164, -0.0070672394, 0.017373372, -0.0026944238, -0.017025903, 0.022312373, -0.025861504, -0.011801483, 0.007811812, 0.007786993, -0.026084876, 0.0061799563, -0.03864334], index=3, object='embedding'), Embedding(embedding=[-0.016797202, 0.009931, -0.008595905, -0.01545553, 0.003554772, 0.039566156, -0.036830198, -0.012246041, 0.0033443135, -0.0049655, 0.02513661, 0.021598281, 0.022453267, 0.024610465, 0.0036238285, -0.0018612404, 0.018586097, -0.010351917, 0.0025107015, -0.0059586, -0.037172195, 0.0008788277, -0.005468627, -0.00560674, -0.010378224, -0.018099412, 0.023097796, -0.021032674, 0.0030417799, -0.027859414, 0.018546635, -0.017178657, -0.00045215644, -0.008931323, -0.0146926185, 0.008516983, -0.0054423194, -0.02495246, 0.020782754, -0.017941568, 0.0048208097, 0.0041664164, 0.0017691649, -0.005445608, -0.0071621584, 0.016126364, -0.008635366, 0.0022739358, -0.0107267955, 0.01830987, 0.027727878, 0.03051645, -0.013916554, -0.03509392, -0.012548574, 0.0005109368, -0.012272349, 0.002370944, -0.008411754, -0.006984584, 0.0024350681, -0.019848846, -0.002842831, -0.016981352, -0.0062348265, 0.0056889504, -0.002490971, 0.011851432, 0.00991127, 0.025504913, 0.04203904, 0.0039592464, 0.008464369, 0.007852725, 0.027096502, -0.0012685044, 0.011831701, 0.006073694, 0.012916877, 0.0016622916, 0.022979412, -0.023558173, -0.007990838, 0.02701758, 0.023808092, 0.017981028, -0.013732403, 0.03170028, -0.026530897, -0.020427607, -0.019927768, 0.028990628, 0.0026964967, 0.0024695962, -0.022597957, 0.014284856, 0.0064189774, 0.0146794645, -0.010121727, -0.019454237, 0.009003668, 0.016889276, -0.02286103, -0.009621889, -0.031516127, -0.008549867, 0.016244749, -0.013535098, -0.021032674, -0.018901784, -0.009746849, 0.0026915642, 0.020756448, -0.032621033, -0.017073428, -0.022545343, -0.014074397, -0.02664928, -0.027070196, -0.03283149, 0.032121193, 0.020203995, 0.029727232, -0.029069548, -0.007523883, 0.012936607, -0.011160865, -0.03051645, 0.0077738026, 0.0063926703, 0.013239141, 0.023137257, 0.0018612404, -0.0035416181, -0.025149764, 0.036988042, -0.039750308, -0.0017329924, -0.025162918, -0.01592906, 0.0032308635, 0.029201087, -0.011331863, -0.0009865231, -0.023479251, 0.005800756, -0.0060605407, 0.016560435, 0.009871809, -0.014574235, -0.0031963352, -0.008609059, 0.032568417, 0.013252295, -0.0022459845, 0.028727554, -0.00184151, 0.008898439, -0.013837632, -0.022413807, -0.0025468739, 0.012127658, 0.024334239, -0.0023939628, 0.027780492, 0.030332299, 0.0117067415, -0.018323023, -0.021361515, 0.013423292, 0.0029020223, 0.02354502, -0.00086402986, 0.0017017524, -0.011594935, 0.021585127, 0.003442966, -0.014074397, -0.028043564, -0.024676234, -0.01942793, 0.011555474, 0.026123134, 0.03643559, -0.011259518, -0.002354502, 0.034541465, -0.036382973, 0.00011108662, -0.023571327, 0.008293372, 0.009029975, 0.0061723464, -0.025662756, -0.66083896, -0.024965612, 0.014140165, 0.00277213, 0.0023808093, 0.05095721, 0.0076422663, 0.010371647, -0.019980382, -0.00533709, 0.0051924004, 0.00092815387, -0.02447893, 0.0002330661, 0.015337147, -0.0121802725, -0.008280218, 0.009062859, -0.029332623, 0.01632367, -0.021940276, 0.005702104, -0.020230303, -0.0018036932, 0.013311486, 0.0002078892, 0.019506851, -0.005478492, -0.012831378, -0.0054357424, -0.011423938, -0.0073200017, 0.02188766, -0.003715904, 0.044327773, -0.013008952, -0.00866825, 0.016994506, 0.011982968, 0.020243455, -0.02257165, -0.013863939, 0.016520975, 0.0016491379, -0.013377254, 0.008484099, -0.005396282, 0.008595905, -0.010259841, -0.013824478, 0.024321085, 0.0010358493, -0.0252813, 0.004636659, 0.0003646025, -0.011673857, 0.038014024, -0.016336823, 0.004363721, 0.018046796, 0.0004141342, 0.0013581136, -0.03614621, -0.011101674, -0.0045708907, -0.0092075495, -0.009240434, 0.008260488, 0.014363777, -0.013627173, -0.002043747, -0.0032160655, -0.0060539637, 0.017717956, 0.013318063, 0.0011788952, 0.036461897, -0.0027967931, -0.027754186, 0.024610465, 0.005396282, -0.018178334, 0.008924746, -0.0022295425, 0.012785341, 0.0009404854, -0.010457146, -0.00042502707, 0.0056757964, -0.02426847, 0.02000669, 0.019598927, 0.027070196, -0.005577144, 0.0038178447, 0.011621243, -0.0011205259, -0.0006950878, 0.0075962283, -0.014219088, 0.01913855, -0.0003740567, 0.029043242, -0.0059618885, 0.039013702, 0.0029809442, -0.014350624, 0.028806476, 0.03711958, -0.0019582484, -0.011022752, -0.024071164, -0.03748788, -0.014758387, 0.00036419148, -0.03098998, 0.00533709, 0.01034534, 0.006945123, -0.00012598722, 0.001193693, 0.014258549, 0.005478492, -0.006142751, -0.0049655, 0.019046474, -0.015074074, -0.041828584, 0.007109544, 0.007971107, 0.017744264, -0.00709639, 0.018612403, -0.04240734, 0.018625557, 0.002826389, 0.01921747, 0.010792564, -0.0046498124, 0.00051381416, -0.028332945, -0.012588035, 0.010568951, -0.018112564, -0.0102006495, -0.010430838, -0.001280836, 0.0009733695, -0.00610329, 0.009503506, 0.0071950424, 0.0050608637, -0.0065735327, 0.02668874, 0.012607766, -0.029727232, 0.0076751504, -0.052088425, -0.0015356877, -0.019362161, 0.005389705, 0.026070518, -0.03080583, 0.015902754, 0.005517953, -0.0089379, 0.0030500009, 0.009635043, 0.0031470088, -0.01531084, -0.009286472, -0.024202703, -0.017560113, 0.018152026, -0.012344693, 0.012265772, 0.0025929117, 0.0055146646, -9.537675e-06, -0.0029266854, 0.002908599, -0.00511019, -0.014797848, 0.027149118, 0.04593252, 0.009391701, 0.026425667, 0.039566156, -0.026241517, 0.013462753, -0.009187819, -0.0043998933, -0.011410785, 0.001930297, 0.009385124, 0.021874508, 0.013535098, -0.00792507, 0.004567602, 0.0046333703, 0.0044426424, -0.0075501907, 0.004255203, -0.0047057155, 0.0041072248, -0.028438175, 0.0020996502, -0.023058334, 0.02946416, 0.021979736, -0.0013909977, -0.01661305, -0.013258872, 0.01607375, 0.010088843, 0.018809708, -0.009542967, 0.0121736955, -0.0054324544, 0.016205287, 0.0062611336, -0.003285122, 0.039250467, -0.0117067415, -0.005156228, -0.012371001, 0.01319968, 0.014232241, 0.033462867, -0.016192134, -0.004590621, 0.016139518, -0.01404809, 0.0043472787, -0.018086258, -0.016468361, -0.011818548, -0.016402591, 0.026780816, -0.006645878, 0.0028658498, 0.020388145, 0.038198177, -8.678321e-05, 0.022624265, 0.0010416041, 0.009878385, 0.0030253378, -0.0013688009, 0.02517607, -0.0055705677, 0.016770894, -0.011022752, -0.026741354, 0.017717956, -0.018362485, -0.01845456, 0.0029497042, -0.0044097584, 0.022703186, -0.012390731, 0.022150734, 0.0059224274, -0.009635043, 0.017625881, 0.02416324, 0.001890836, -0.015587066, -0.03306826, -0.0020404588, -0.013824478, -0.008858978, 0.01585014, 0.0047287345, -0.007918493, -0.010351917, 0.02567591, -0.018415099, 0.013956014, 0.02054599, -0.009424585, -0.01816518, 0.012029006, 0.020493375, 0.0052187075, -0.016994506, -0.007898762, 0.039855536, -0.011568628, 0.011371324, 0.009753426, 0.008589328, 0.02466308, 0.0017033967, 0.024636772, 0.01307472, 0.010391377, -0.026872892, -0.0013112538, -0.006622859, -0.0066261473, 0.014271702, -0.017981028, -0.006231538, 0.01834933, 0.009378547, -0.0040250146, 0.004271645, 0.012732726, -0.027280655, 0.008609059, 0.004866848, -0.03914524, 0.016415745, 0.008589328, 0.007267387, -0.013219411, -0.013219411, 0.026031058, 0.0013367389, -0.012535421, -0.013101027, -0.032489497, 0.006208519, 0.07597544, 0.05098352, 0.0033443135, 0.009878385, -0.008951054, 0.017310193, -0.009154935, -0.04164443, 0.016810356, -0.024492081, 0.003636982, 0.005123344, -0.0063433438, 0.0038309982, -0.01607375, -0.011154288, -0.0006696026, -0.0018316447, -0.0014395018, -0.009043129, -0.03275257, 0.014153319, 0.026451975, 0.02098006, 0.01834933, -0.009865232, 0.012509114, 0.015968522, -0.0020536124, -0.006514341, 0.0075830747, 0.028122487, 0.025938982, 0.0096876575, 0.008398601, 0.027201733, -0.0063926703, -0.0101874955, 0.027070196, -0.0007082414, 0.017520651, 0.012107927, 0.0004854516, -0.015600219, 0.009154935, 0.0038573055, -0.035304375, 0.030148149, -0.0035383299, -0.015613373, 0.029095856, -0.0056494894, -0.011331863, -0.025662756, -0.00765542, 0.034436237, 0.0069253924, -0.021374669, 0.015481837, -0.016415745, 0.0010810649, -0.0053338017, 0.006238115, -0.015639681, -0.018809708, -0.020966906, -0.015205611, -0.0014189491, -0.02578114, -0.019164857, 0.0020421029, -0.012555151, -0.01921747, 0.009845501, 0.014797848, 0.00085005414, 0.018546635, 0.0023528577, -0.0061230203, 0.016086904, -0.017020812, -0.020164534, 0.023637095, -0.020059304, -0.012811648, -0.009753426, 0.0076356893, -0.02058545, -0.0036731546, 0.027491113, 0.013114181, 0.0016351622, -0.004140109, 0.0036600009, 0.0035613487, 0.0056034517, 0.015942214, 0.008747173, -0.019059628, -0.010253265, 0.0054752035, -0.018717634, -0.0004981942, -0.02050653, 0.028648634, -0.0021687069, 0.0060901362, 0.006471592, -0.034146857, 0.01264065, 0.005389705, 0.009431162, 0.01570545, -0.013022106, 0.002242696, 0.040565833, 0.018099412, 0.014745233, -0.01276561, -0.01433747, 0.00575143, -0.052193653, 0.031305667, 0.027464805, 0.013574559, 0.028464481, 0.016244749, -0.032936722, 0.007859301, 0.021216825, 0.016784048, -0.021821892, -0.022597957, 0.00049531687, -0.036672354, -0.01661305, -0.013265449, 0.02914847, 0.0074186544, -0.014298009, -0.016257903, -0.007050352, -0.012528844, -0.016876124, -0.009148358, -0.025623295, -0.011904046, 0.009181242, -0.0013022106, 0.055718828, -0.0017099734, 0.019204319, -0.012673534, -0.00019298859, 0.007392347, -0.029227393, -0.010457146, 0.0097008115, 0.03185812, 0.0064189774, 0.018980706, 0.01715235, 0.030042918, 0.0117001645, 0.0028790035, 0.011167442, -0.010016498, -0.0075962283, -0.027149118, 0.005767872, 0.017073428, 0.008457792, 0.016547281, -0.012851109, 0.01990146, 0.01723127, 0.011489706, -0.024834076, 0.00080155005, -0.03730373, -0.017599573, 0.007859301, 0.0037389228, -0.008267065, 0.00115341, -0.011759357, 0.014100704, 0.011529167, 0.035883136, -0.0030894617, 0.027833108, -0.026070518, 0.015915908, -0.030200763, 0.019020166, -0.02354502, 0.0035975212, 9.197273e-05, -0.0007135851, 0.026886044, -0.009733696, -0.0073200017, 0.012430192, 0.008003991, -0.010687334, 0.0017313481, -0.020453913, -0.02134836, -0.007497576, -0.02180874, -0.004824098, -0.01907278, -0.017494345, 0.016468361, 0.020269763, -0.0078001097, -0.023492405, 0.019625233, -0.018586097, -0.027149118, -0.02120367, 0.0012323319, 0.016455207, 0.016955044, 0.026872892, 0.03477823, 0.0039526694, -0.019940922, 0.0068662013, -0.011759357, -0.006606417, 0.030279685, 0.049431387, -0.020519681, -0.025018228, -0.0011205259, -0.00736604, -0.004916174, -0.010424262, -0.012989222, 0.0064518615, 0.015613373, -0.0070371986, 0.0074055004, -0.011233211, 0.008780057, -0.010766257, -0.0030302703, -0.0035580602, -0.009510083, -0.036304053, 0.0050904597, -0.008056606, 0.024321085, -0.013357524, -0.002466308, 0.0019368738, -0.031358283, -0.025412837, -0.0024728847, 0.006645878, 0.009319356, -0.03409424, -0.004146686, 0.00021929588, 0.018152026, -0.011325286, 0.0013276958, -0.0033903513, 0.01690243, 0.0014493669, -0.0008369005, -0.00546205, -0.02838556, 0.012680111, 0.010042806, -0.0071161203, -0.015902754, -0.016152672, 0.0022311865, 0.021953428, -0.0051134783, 0.011594935, 0.015468683, 0.0058040447, -0.023321407, 0.0034298121, -0.0031289225, -0.0049128854, 0.0032834779, -0.006017791, -0.0010851754, -0.013561405, -0.008655096, 0.015337147, -0.0034988688, 0.019664695, 0.019598927, -0.032279037, 0.00056190713, -0.0178758, -0.0060769827, -0.026596665, 0.028727554, -0.025123456, -0.018770248, 0.004015149, -0.008431485, -0.028306639, -0.016823508, 0.0021423995, 0.017902108, 0.010358493, 0.0003580257, 0.02350556, 0.013521944, 0.011042482, 0.0031338553, -0.035014994, 0.0082078725, -0.017165503, 0.0008697846, -0.022466421, -0.0014542995, 0.002602777, -0.03485715, -0.0014115502, 0.025583833, -0.015442376, -0.033962704, 0.0009265097, 0.031121518, -0.005215419, -0.007727765, -0.013640327, 0.0010448925, -0.008865555, 0.028148795, -0.010161188, 0.0075107296, 0.017560113, 0.021032674, 0.029306315, 0.012976068, -0.0073331557, -0.018375639, -0.005215419, -0.02639936, -0.03777726, -0.021835046, -0.0063696513, 0.011377901, 0.021532513, -0.0126866875, -0.037145887, -0.011739626, -0.023716018, -0.014745233, -0.01264065, 0.0024202701, 0.0038573055, 0.0034890035, -0.020374991, 0.011134558, -0.0017740974, 0.035567448, -0.026622972, -0.034278393, 0.0010021431, 0.0065965517, 0.011943507, -0.044485617, 0.005248303, -0.0021374668, 0.000344461, 0.021874508, 0.0053995703, 0.017060274, 0.007569921, -0.0055935862, -0.01247623, 0.019848846, -0.010568951, 0.012436769, -0.0044064703, -0.016139518, -0.012864262, 0.006991161, -0.015665988, -0.025360223, 0.007865878, 0.02134836, -0.013127335, -0.009049706, 0.0034396774, -0.0003820722, -0.006497899, 0.008497253, 0.024281623, -0.034541465, -0.00554426, 0.009582428, 0.010687334, -0.0258206, -0.011509437, 0.009470622, -0.037803568, -0.004196012, 0.01783634, -0.031937044, 0.0053601093, -0.023623941, 0.014376931, -0.011279249, -0.008845825, -0.028990628, -0.0029562812, -0.013390408, -0.0112003265, 0.027912028, -0.034436237, -0.011246365, -0.0141664725, -0.0076291123, -0.004626794, -0.007931646, 0.007931646, -0.025333915, -0.012022429, 0.02286103, -0.023452945, -0.015337147, 0.019940922, -0.011075367, 0.034331005, 0.018283563, 0.21024781, 0.010601835, 0.008405178, 0.034620386, 0.0017412134, 0.02054599, 0.014284856, -0.0023775208, -0.009161511, 0.03109521, -0.019875154, 0.0021177365, 0.002019084, -0.0008105932, 0.0029233969, 0.0055870097, -0.029174779, -0.0004912063, -0.038408637, 0.002499192, -0.024623618, -0.012666957, -0.0032177097, -0.006320325, 0.041118287, -0.00922728, -0.024215855, 0.00951666, -0.0049523464, 0.004413047, -0.018875476, 0.0112069035, 0.0060835597, 0.006793856, -0.013535098, -0.016586743, 0.01805995, -0.011759357, 0.02751742, 0.0043209717, 0.030069226, -0.007247657, -0.021243133, -0.007352886, -0.001969758, 0.0018168468, 0.014113858, -0.016165826, 0.014587389, 0.012324963, -0.0030795964, -0.030647986, 0.007694881, 0.0060835597, -0.010023075, 0.01777057, 0.021322055, 0.0121802725, -0.02217704, -0.004202589, -0.027570033, 0.03661974, 0.021295747, 0.019664695, -0.0025189226, 0.0049457694, -0.02513661, 0.011673857, 0.011035906, -0.011220057, -0.0017806743, -0.014824155, -0.011423938, 0.020848524, -0.015363454, -0.023531865, 0.020282917, 0.012015852, -0.0074383845, 0.030542757, -0.0070766597, 0.011555474, 0.0055902977, -0.005981619, -0.012522267, -0.038487557, 0.017954722, -0.014981998, -0.03809295, -0.02304518, -0.012436769, -0.030384913, -0.018336177, 0.017218119, 0.013304909, -0.0018546636, 0.01193693, 0.018230949, -0.013068143, 0.0037882489, -0.028332945, 0.02983246, 0.011305556, 0.023518713, -0.012976068, -0.019204319, 0.022045504, 0.005116767, 0.017244425, -0.009128627, -0.011226634, -0.008799787, 0.022650572, -0.008714288, 0.027175425, 0.0057218345, 0.0067741256, -0.023952782, 0.010463722, -0.015810678, 0.022242809, -0.017717956, -0.00017099734, 0.008161835, -0.010818871, -0.017849492, -0.0007111188, 0.006606417, 0.011798817, -0.046537586, 0.016494667, 0.031884428, 0.00808949, -0.025873214, 0.004038168, 0.0028938013, 0.027727878, 0.011062213, -0.009996768, 0.03554114, 0.014613696, -0.010522914, 0.022374345, -0.00027047176, -0.01617898, -0.011292402, 0.01777057, -0.0146794645, -0.008240757, -0.03980292, -0.0055870097, -0.026031058, -0.03514653, -0.008845825, 0.041618124, 0.0046037748, -0.036567125, -0.0278068, -0.011055636, -0.007971107, -0.040250145, 0.0102006495, 0.017020812, -0.003995419, -0.012384154, 0.0029908095, -0.16910322, 0.042880874, 0.015429222, 0.009029975, 0.016652511, 4.829853e-05, 0.03798772, 0.0017872511, -0.0016885988, 0.009549544, -0.0008381336, -0.0010933966, -0.010647873, 0.00044352436, 0.007458115, -0.0017740974, -0.020138226, 0.02484723, 0.023400329, 0.015521298, 0.015994828, -0.023768632, 0.022611111, -0.003732346, -0.0034067933, 0.015481837, -0.020887984, 0.011818548, -0.029280007, 0.002458087, -0.017494345, -0.009608735, 0.0361199, -0.00024046503, -0.00475833, -0.018152026, -0.019283239, -0.0075764977, -0.022703186, 0.011890893, -0.01077941, 0.03759311, 0.0015759708, 0.002763909, -0.0054587615, 0.0131865265, 0.019388469, -0.0048701363, 0.012502537, -0.025807446, 0.017625881, -0.036856506, 0.025333915, 0.0036106748, -0.0092009725, 0.0010720219, 0.004534718, 0.017810032, -0.0040677637, -0.016428899, -0.011673857, -0.021611433, 0.007471269, 0.0023840975, 0.0053929933, -0.032673646, -0.026636126, -0.014245395, -0.012338117, 0.018152026, 0.017599573, -0.040802598, -0.0038869013, -0.015600219, 0.0073463093, 0.008674827, -0.018651865, -0.012594612, -0.018572943, -0.011568628, -0.022584803, 0.018717634, -0.025333915, 0.011220057, -0.031147825, 0.011548898, 0.0015677498, 0.026438821, -0.0020700544, 0.0033870628, 0.00376523, 0.0014847174, 0.008858978, -0.017481191, 0.032226425, 0.003932939, 0.014837309, 0.023413483, -0.002809947, 0.009036552, 0.01177251, -0.013087874, -0.010878062, 0.0012923453, 0.011831701, 0.0020141515, -0.010240111, 0.0067872796, 0.018388791, -0.01346933, -0.008516983, 0.009457469, 0.007050352, 0.016731434, -0.0029842325, 0.0048504057, 0.004245338, -0.02350556, -0.00908259, 0.009845501, 0.06939862, 0.0034002166, 0.004768195, 0.016770894, -0.010884639, 0.030963674, -0.060401525, -0.018783402, 0.0146926185, 0.025544373, 0.014968845, 0.01274588, 0.0046563894, 0.0087077115, -0.023597633, 0.03614621, 0.0027441785, 4.3674204e-06, -0.0020059305, 0.0040480336, 0.01884917, -0.019270087, -0.011489706, -0.02946416, -0.0258206, 0.016626203, -0.011634396, -0.002540297, 0.017625881, -0.006543937, -0.019296393, 0.0015989896, -0.012469653, 0.0238344, 0.019796232, -0.01448216, 0.012272349, -0.009812617, 0.024400007, -0.029885076, -0.00753046, 0.007208196, 0.0024087608, -0.025347069, 0.016139518, -0.02993769, 0.016139518, 0.0028296772, -0.007010891, -0.021427283, -0.015323993, -0.0045610256, -0.0022574938, 0.0029020223, 0.0028576287, -0.005139786, -0.003459408, -0.0021818604, -0.004005284, -0.014547928, 0.012107927, -0.02069068, 0.0025846907, 0.026675586, -0.027359575, -0.015074074, 0.0213089, -0.00072221714, -0.024978766, 0.011627819, 0.0045708907, -0.0107136415, -0.022874184, -0.01164755, 0.023321407, -0.0072279265, -0.0011706741, 0.012548574, -0.030463835, -0.00011643029, -0.017073428, 0.005439031, -0.015679142, -0.0071226973, -0.009345663, -0.029280007, -0.0063170367, -0.030700602, -0.0019516717, -0.0059947725, 0.013614019, 0.001009542, -0.014298009, -0.0053502438, 0.017454883, -0.029858768, -0.0014353913, 0.01332464, 0.027964644, -0.011167442, -0.023466097, -0.011312133, -0.0041664164, -0.010062536, -0.0035416181, 0.0062183845, -0.038803246, -0.00037528985, -0.072502874, 0.029858768, -0.0005384772, 0.004445931, -0.019520005, -0.005619894, 0.0014321028, -0.008082913, 0.007984261, 0.023926476, -0.005198977, 0.017757418, 0.016442053, -0.007786956, -0.022387499, -0.045485295, 0.034646694, 0.011976391, 0.03606729, 0.032279037, 0.009628466, -0.010614989, -0.0065077646, 0.0048964433, -0.0053338017, 0.019296393, -0.023518713, 0.004692562, -0.0035514834, -0.0065406486, 0.0011665636, -0.031831816, -0.0077080345, 0.014745233, -0.013640327, -0.0361199, 0.0107070645, 0.014824155, -0.006182212, -0.010634719, 0.009608735, -0.01220658, 0.00035946438, -0.0009585717, -0.019546313, 0.019730464, -0.025649602, -0.007050352, 0.018046796, -0.0066327243, 0.014600543, 0.019007012, -0.008740596, 0.00014838953, 0.013133911, -0.028806476, 0.022782108, -0.011345017, 0.0021259575, -0.00467612, 0.022334885, 0.0067083575, 0.019388469, 0.013239141, 0.01009542, 0.0045380066, -0.017678495, -0.0005035379, 0.022058658, -0.027175425, -0.019506851, 0.008720865, 0.020887984, -0.00575143, 0.008780057, 0.00866825, -0.008905016, -0.0021539088, -0.008516983, 0.03980292, 0.03043753, 0.008780057, -0.0027096502, 0.0029151759, 0.017954722, 0.03254211, -0.023808092, -0.007142428, 0.013600866, 0.0007111188, -0.01592906, 0.0139034, -0.021571973, -0.00013759943, 0.012574882, -0.014587389, -0.0012956337, -0.01777057, 0.0076751504, 0.025202379, 0.020519681, 0.008872132, -0.010838601, 0.0032341518, -0.028464481, 0.006760972, 0.013101027, -0.019625233, 0.0037290575, 0.011831701, 0.01462685, -0.011529167, -0.01671828, 0.034541465, -0.034567773, 0.0046103518, -0.0004669543, 0.010983291, -0.02174297, 0.016271055, 0.0058829663, -0.01346933, -0.008549867, -0.040144917, 0.009733696, 0.02249273, -0.00937197, 0.0014222376, 0.018152026, -0.006024368, 0.0001490061, -0.007523883, 0.02751742, -0.013850785, -0.0059586, 0.01859925, -0.018783402, 0.012285502, 0.0024071166, 0.06803064, 0.007846148, -0.0055902977, -0.012818225, 0.019414775, 0.013890246, 0.00953639, 0.0076751504, -0.0072936947, -0.015363454, 0.01899386, 0.011371324, -0.008115797, -0.010266418, -0.030200763, 0.008457792, -0.010930677, -0.0019039897, 0.00023121637, -0.0013507146, 0.020572297, 0.0053995703, 0.01744173, -0.0006317859, -0.042670414, -0.007504153, 0.004748465, 0.0021802161, -0.0107136415, -0.019862, 0.012831378, -0.0096876575, -0.0023758765, -0.01133844, 0.021374669, -0.029043242, -0.018257255, -0.014140165, 0.016876124, 0.014311163, -0.049536616, 0.02397909, -0.013107604, -0.006728088, 0.020638065, -0.012962914, 0.007497576, -0.016639357, -0.012252618], index=4, object='embedding'), Embedding(embedding=[-0.011725959, 0.006645145, -0.00589557, -0.018524278, -0.022161346, 0.02987871, -0.025993956, -0.01428755, 0.0066092955, -0.0066516628, 0.008180144, -0.020636125, 0.012214812, 0.011576044, -0.008375686, 0.012006235, 0.010454941, -0.034597773, 0.011654261, -0.0142223695, -0.008382203, 0.025159646, 0.009900907, 0.00045137448, -0.019123938, 0.013296808, -0.0049341586, -0.011732478, 0.010670036, -0.017742114, -0.00449745, -0.028966183, -0.019840922, -0.031703763, 0.0049146046, -0.0034089366, -0.020909881, -0.030791236, 0.029930854, -0.0038586815, 0.006052003, -0.007215474, -0.0031498442, -0.03517136, -0.0152913295, 0.011530418, -0.0015871435, -0.0030227425, -0.008030229, 0.02987871, 0.005752173, 0.0123908, 0.0009638556, -0.019840922, 0.011999717, -0.0031628804, -0.025967883, -0.0018152749, -0.0059737866, -0.00065098953, -0.007287172, 0.003614255, -0.040020786, 0.01155649, -0.022826187, -0.031521257, -0.020375403, 0.00038619406, 0.0016213632, 0.026802193, 0.031182319, 0.0037120255, 0.0157867, -0.014704705, 0.004777726, -0.020922918, 0.0033730874, 0.010089931, -0.00118221, 0.0058434256, 0.011719441, -0.038899682, -0.018054979, 0.021965805, 0.021522578, 0.019058758, 0.0073523526, 0.020531835, -0.0140529005, 0.0004876311, -0.003607737, 0.018706784, 0.010526639, 0.015069716, 0.0038782356, 0.005797799, -0.011895428, 0.03047837, -3.7332347e-06, -0.012162669, 0.011823731, 0.015943134, 0.0057000285, -0.01059182, -0.032016627, -0.010070376, -0.009868317, -0.006505007, -0.004090072, 0.0044713775, -0.010630928, -0.01235821, 0.009711884, -0.044348765, -0.009379464, 0.0025860337, -0.015643304, -0.0061106654, -0.0140920095, -0.007919422, 0.03120839, 0.03373739, 0.003027631, 0.002268279, 0.017833365, 0.024520878, -0.020205934, -0.019997356, 0.012514642, 0.0064528626, 0.026463255, 0.0016881732, 0.022148311, 0.012319101, -0.02208313, 0.009953052, -0.013342435, 0.027297564, -0.04583488, -0.038743246, -0.004093331, 0.012527679, 0.004725581, 0.015604195, 0.015226149, 0.026958626, 0.010396278, -0.0078020976, 0.016933875, -0.014365767, -0.0018674193, -0.025146611, 0.024651239, 0.008916683, 0.00022018765, 0.029200833, 0.022774043, 0.026033064, 0.0034187136, -0.014313622, -0.0066516628, 0.026658796, 0.017637825, -0.037126772, 0.005185103, 0.0043051676, 0.0024328595, 0.015317401, -0.023334594, -0.021300964, -0.0055533727, -0.012645003, -0.04140261, 0.007065559, -0.0010485902, 0.02188759, -0.005765209, 0.022552429, -0.026163425, 0.0015757369, -8.463272e-05, 0.0066809943, 0.0027408372, 0.046590973, -0.014874174, 0.03564066, 0.010135557, 0.002727801, 0.0050938507, -0.025120538, 0.004536558, 0.0061302194, 0.020336295, -0.002647955, -0.6803794, -0.018367846, 0.012697148, -0.02797544, 0.0041454756, 0.018263556, 0.019906104, 0.0051720673, -0.021261856, -0.006358351, -0.023152089, 0.01918912, 0.0050221523, -0.006606037, -0.003813055, 0.0005312205, 0.00816059, -0.006658181, 0.03188627, 0.013075194, 0.015095788, 0.027792936, 0.0012082822, -0.0038097962, 0.018654639, 0.016438505, 0.018576423, -0.043384094, -0.02527697, 0.00016468244, 0.005175326, 0.026723977, 0.019462876, 0.013531458, 0.046304177, 0.0058303894, -0.014196298, 0.010376724, 0.016503684, 0.025759306, 0.003467599, -0.014509164, 0.0038260913, -0.005315464, -0.016295107, 0.0077825435, 0.012051862, -0.014939355, -0.02218742, -0.0056902515, 0.0077499533, -0.014013792, -0.012762329, 0.0002283352, 0.0074892314, -0.0063616103, 0.02221349, 0.013974684, 0.014535236, -0.009887871, -0.00054833037, 0.009705367, -0.021105424, -0.0064300494, -0.018980542, 0.012325619, 0.0023839741, -0.004738617, 0.013466277, 0.0034350087, 0.00286468, 0.0389779, -0.025042322, -0.0031954707, 0.01349235, 0.034519557, 0.027949369, -0.0026039581, 0.0005202213, 0.01931948, 0.017129418, 0.023725677, 0.0023807152, -0.010630928, 0.018041942, -0.0022161347, -0.01931948, -0.014639525, 0.029722277, 0.01055923, 0.0052535427, -0.0054099755, -0.0054816743, -0.026202533, -0.0023562727, 0.014717741, -0.0008106816, 0.046304177, 0.014769886, -0.022461176, -0.027088987, -0.0022601315, -0.009307765, 0.026593616, -0.0049634897, 0.009431608, 0.011080673, 0.009979124, 0.028027585, -0.022057058, 0.0014958909, -0.005537078, -0.02128793, -0.006765729, 0.0071502933, -0.0285751, 0.018550351, 0.0026756567, 0.025120538, -0.0011944314, -0.0015684041, -0.013909504, 0.014535236, 0.0070460048, 0.014261479, -0.015134896, 0.0074501233, -0.0016995798, -0.005915124, 0.0016254369, 0.008668997, -2.164194e-05, 0.027949369, 0.004839647, 0.013779143, 0.0017500946, 0.016112603, -0.022852259, 0.017663896, 0.0116347065, -0.018002834, 0.005178585, 0.01368789, -0.015017571, -0.0031824345, -0.018785, -0.015512942, -0.0021004393, -0.020597016, 0.012338655, -0.009229549, -0.0033160544, -0.021261856, 0.015799737, -0.019397696, -0.020153789, 0.013081713, -0.0070199324, 0.0014926319, -0.010702627, 0.010200737, 0.016959948, -0.022030985, 0.0032720575, 0.014378803, -0.0063029476, 0.015278293, 0.0015056679, -0.0037250617, -0.034649916, -0.0064039775, -0.0060226717, -0.013120821, 0.0131729655, -0.021405254, 0.013896468, -0.007137257, -0.009053562, 0.00379676, -0.0024263416, 0.0039923014, 0.010930758, -0.01871982, -0.0014608564, 0.02807973, -0.003174287, 0.011628189, -0.004256282, -0.005188362, -0.010696108, 0.004526781, 0.0060031177, -0.00036949158, 0.004376866, 0.01332288, 0.010924241, -0.024364445, -0.0024621908, -0.016438505, 0.02831438, 0.018433025, 0.005009116, 0.0015276663, -0.00023302004, 0.023373703, -0.040881164, -0.009783583, -0.02328245, 0.03000907, -0.004670178, -0.0063713873, -0.024364445, 0.013596638, -0.003203618, -0.014704705, 0.034597773, -0.024820708, 0.016959948, 0.012162669, 0.011738996, 0.009588041, 0.021248821, 0.0049048276, -0.010278954, 0.00041674738, -0.002714765, 0.0042823544, 0.021405254, -0.004393161, -0.013544493, -0.010761289, 0.0039890422, 0.010787361, 0.013713962, 0.021157568, 0.03816966, 0.0145221995, -0.02688041, 0.019958248, 0.008779804, 0.00349693, 0.017298887, 0.027349709, -0.019202154, -0.0050742966, 0.00722851, 0.024468733, 6.5536886e-05, 0.011523901, 0.028366523, -0.0052046576, 0.0018934915, -0.008988381, -0.007919422, 0.013427169, 0.0116347065, -0.0008241251, 0.000394749, 0.01095683, 0.03707463, -0.0018331996, 0.006465899, 0.022578502, -0.011354431, -0.0084473835, 0.0015610713, 0.017963726, -0.0068895714, 0.0125341965, -0.00050229667, 0.008069337, -0.0040248916, 0.004357312, -0.012801437, -0.0073784245, -0.011830248, -0.01095683, -0.011908465, 3.1597232e-05, -0.013414132, -0.028601173, -0.037257135, -0.00699386, 0.018341772, 0.0002957562, 0.004080295, -0.02298262, -0.005735878, -0.0010673295, 0.0029314898, -0.009131778, 0.010995938, -0.011725959, -0.014483091, -0.005325241, 0.009470716, 0.03783072, -0.03133875, -0.008519082, -2.7294305e-05, -0.0011088821, -0.0023236824, 0.004243246, -0.00082860625, 0.018941432, -0.010142075, -0.014183261, -0.008564709, -0.004706027, -0.011282733, -0.0030520738, 0.023491027, -0.023634424, 0.026528435, 0.009438126, -0.012104006, -0.02081863, 0.012325619, 0.043905538, 0.0007886832, 0.008251842, -0.016764406, -0.042419422, 0.008147554, 0.08676819, -0.014209334, -0.007065559, 0.003627291, -0.011439166, 0.010305027, -0.018693747, -0.005869498, 0.01188891, -0.00020766078, -0.0011145853, -0.01788551, 0.019241262, -0.0008000898, 0.025641982, -0.0031465853, 0.005416494, 0.003757652, 0.007156811, 0.011497828, -0.018576423, 6.706455e-05, -0.009509825, -0.0016009944, 0.0265154, -0.0026267713, 0.01059182, 0.017729077, -0.0051329588, -0.01215615, -0.0023839741, 0.0042986493, 0.002517594, -0.00063102803, -0.008871057, 0.011914982, -0.021522578, 0.018941432, 0.02701077, -0.01961931, 0.034154546, 0.016568866, -0.004487673, -0.021692047, 0.022878332, -0.009112224, 0.012247403, 0.0041715475, 0.016373323, -0.013120821, 0.017950691, -0.012625449, 0.01139354, -0.030739091, 0.0019831145, 0.003327461, -0.002268279, 0.009920461, -0.032120917, -0.026854338, -0.006358351, -0.025563765, 0.00029168243, -0.00017160786, -0.019997356, -0.029018328, -0.015512942, 0.0082779145, -0.019397696, -0.010363689, 0.023673533, -0.017051201, -0.0034904121, 0.011849802, 0.019554129, 0.022461176, 0.019449841, -0.00366314, -0.0067005483, 0.04825959, -0.01801587, -0.021705084, -0.008786322, 1.5785885e-05, -0.008486492, 0.028549029, 0.004842906, -0.035197433, -0.01119148, 0.014509164, 0.011563009, -0.0072024376, 0.01349235, -0.0145221995, -0.005116664, -0.013596638, 0.003777206, -0.002118364, -0.021327037, 0.017155489, 0.0021362887, -0.0140920095, 0.022800114, -0.005592481, 0.006377905, 0.0058597205, 0.00729369, 0.011986681, -0.02411676, -0.00509711, 0.0142223695, -0.016777443, 0.017442284, -0.0030765163, -0.018563386, 0.034910638, 0.02448177, 0.005869498, -0.0029037881, -0.020727377, -0.00885802, -0.020166826, 0.013120821, 0.037100703, -0.006967788, 0.012299547, 0.010728699, -0.008154072, -0.01621689, -0.005165549, -0.014274514, 0.0071698474, -0.00652782, -0.021027207, -0.013909504, -0.018576423, -0.010050822, 0.0128340265, 0.010806915, -0.019462876, -0.013120821, -0.0098161725, -0.010806915, 0.005605517, -0.030634802, -0.025250899, -0.015539015, -0.020036465, -0.001411971, 0.038300022, -0.0034056774, 0.02081863, 0.003457822, 0.010187701, -0.0010005196, -0.011426129, -0.01931948, -0.016503684, 0.023060836, 0.0028418668, 0.04043794, 0.023269413, 0.032616287, -0.0019081571, -0.011054601, -0.0019032686, 0.015330438, 0.0046408465, -0.019880032, 0.005087333, 0.012716702, 0.023869073, -0.009464199, 0.018146232, -0.005387163, 0.014965427, 0.010344135, -0.007443605, -0.0046245516, -0.009731438, -0.010168147, -0.00965974, -0.007554412, 0.01338806, -0.029748349, -0.014417911, 0.03587531, 0.0036012188, -0.007384943, 0.005387163, 0.014782921, -0.01771604, 0.026541471, 0.006338797, -0.002328571, -0.010018232, 0.0018804554, -0.02568109, -0.00039413793, 0.027506141, 0.018367846, -0.00034464154, -0.013635746, 0.011719441, 0.0026789156, 0.021652939, -0.007965049, -0.006746175, -0.0072610998, -0.015604195, 0.009926979, -0.0029233424, -0.010181183, -0.041845836, 0.0030406672, 0.01771604, -0.009575006, 0.022969583, -0.051935766, -0.03936898, -0.018276593, -0.018654639, 0.025263935, -0.0076391464, 0.02148347, 0.030217648, -0.010076894, -0.0019945211, 0.016660118, -0.014939355, -0.010702627, 0.028627245, -0.001984744, -0.011022011, -0.024964105, 0.0047809845, -0.0039857836, 0.019827887, -0.009353392, 0.016034385, 0.023895146, 0.013818251, -0.027819008, 0.008310505, -0.013427169, 0.015056679, -0.008831948, -0.020023428, -0.024898924, -0.004370348, -0.02508143, 0.010963349, -0.00366314, 0.009151332, 0.01848517, -0.031964485, 0.020740412, -0.004409456, -0.009425091, -0.0046408465, 0.008955792, 0.016412433, -0.0057423958, 0.015200077, -0.017742114, 0.021965805, -0.03060873, -0.003157992, 0.0043214625, 0.015369546, -0.024638202, -0.020140752, 0.011113264, 0.02188759, 0.023256378, 0.043253735, -0.009737956, -0.020075573, -0.020610053, -0.015838845, 0.009007935, 0.013583601, 0.0042986493, -0.015591159, -0.005905347, -0.031547327, -0.025263935, 0.004275836, 0.025824487, -0.01188891, -0.023543172, -0.00885802, 0.015539015, -0.009418572, -0.0010404426, 0.0151609685, 0.019971283, 0.015174004, -0.016347252, 0.0038228324, 0.018446062, 0.017559608, -0.033346307, -0.0055207824, 0.009444645, 0.0012742775, 0.005269838, -0.009835727, -0.0008929719, -0.01488721, 0.01608653, 0.013883431, -0.003353533, 0.0028988996, -0.0006807281, 0.009627149, -0.011139336, -0.0015635156, -0.0151218595, 0.012612414, -0.019293407, -0.015695447, -0.011875874, 0.0049341586, 0.011875874, -0.009790101, -0.012117042, 0.017794257, -0.012019271, -0.014378803, 4.9623693e-05, 0.020975063, -0.014665597, -0.020753449, -0.035510298, 0.0011585822, 0.00453004, 0.0311041, -0.0031107361, -0.026606651, -0.012742775, 0.005944455, -0.002048295, 0.02461213, 0.0068830536, -0.006817873, -0.018302664, -0.030791236, -0.036318537, -0.02281315, 0.0043312395, 0.020805594, -0.0018462357, -0.016581902, -0.011243625, 0.0071307393, -0.05292651, -0.019293407, -0.005276356, -0.0021134755, 0.021353109, 0.004989562, -0.00846042, -0.0055729267, 0.011132818, 0.02068827, -0.0006298059, 0.01188891, 0.019136975, -0.0065799644, 0.011582563, 0.004859201, -0.015956169, -0.003786983, -0.016295107, 0.0018771964, -0.0073784245, 0.008284433, -0.007736917, -0.009105707, 0.0021265117, -0.017403174, -0.018511243, 0.0043638297, 0.0097966185, -0.023034764, -0.013935576, -0.009731438, -0.013120821, 0.010930758, 0.01049405, -0.0014176744, -0.0037706879, 0.004116144, -0.0143918395, -0.007717363, 0.0039988193, -0.012253921, 0.01428755, -0.0044355285, -0.010689591, 0.0145221995, -0.011471756, 0.009933498, -0.008362649, -0.004820093, -0.020896846, -0.020649161, -0.0006852093, -0.019071793, 0.0014429318, -0.011960609, -0.00819318, -0.005286133, 0.0071894014, -0.009333838, -0.032746647, 0.0034187136, 0.01488721, 0.008408275, -0.031521257, -0.009724921, -0.020492727, -0.0123908, 0.010819951, -0.016673153, -0.015239185, -0.021092387, -0.009086152, 0.04077688, -0.002367679, -0.033945967, 0.0025257417, 0.012566787, -0.016868696, 0.02608521, 0.2213006, 0.0026169943, -0.005866239, 0.019163046, 0.0067005483, 0.02058398, 0.013635746, 0.0031465853, -0.008297469, -0.0077303993, -0.040907238, -0.017142454, 0.01299046, 0.0065115253, 0.014717741, -0.020675233, -0.035301723, -0.023504063, -0.008551672, -0.013700927, 0.0014241925, 0.0021265117, -0.011660779, -0.012820991, 0.01415719, 0.008590781, -0.006055262, 0.012508125, -0.001442117, 0.023008691, -0.0052209524, 0.0033926414, 0.012064897, 0.0143527305, 0.00079194224, -0.017976763, 0.025863595, -0.007254582, 0.030869452, -0.0008905276, -0.005788022, -0.012899207, -0.011908465, -0.009653222, 0.00449745, 0.02101417, -0.0041454756, -0.020440582, -0.0034187136, -0.0061399965, -0.022239562, -0.016542794, 0.0031254017, 0.019006614, 0.015656339, -0.013818251, 0.022552429, 0.01668619, -0.0056544025, 0.009333838, 0.016712263, 0.028053658, 0.004855942, 0.0054001985, -0.022395996, 0.010709145, -0.0090470435, 0.013120821, 0.026189497, -0.03368525, 0.0053447955, -0.0046017384, -0.004970008, 0.0076391464, -0.018237485, -0.023934254, 0.014795958, 0.052978653, 0.009203477, 0.022330815, -0.032798793, 0.008388721, 0.023790857, -0.005269838, 0.004637588, -0.021001134, 0.0046082563, 0.016229928, -0.003624032, -0.018433025, -0.0005312205, -0.02171812, 0.007287172, -0.009940016, 0.027167203, -0.00083349476, 0.011602117, 0.029852638, -0.012110524, 0.0065734466, 0.0030651097, 0.005439307, -0.0047223223, -0.015760629, -0.025668053, -0.008910165, 0.023477992, 0.02201795, 0.0062899115, 0.01145872, -0.0071502933, -0.03743964, 0.0019570424, -0.014808994, 0.03180805, 0.0007850168, 0.0021868034, -0.02891404, 0.005784763, -0.009333838, 0.0027962404, -0.01921519, -0.00019421731, 0.023008691, 0.023947291, -0.018732855, -0.02071434, -0.025459476, 0.0035653696, -0.010272436, -0.004425751, -0.00043670888, 0.00669403, 0.0116151525, -0.029670132, 0.022852259, 0.0074957497, -0.009275176, -0.020701304, -0.002041777, 0.009913944, 0.0012783512, 0.009262139, -0.020935955, -0.013361989, -0.018276593, 0.008245325, -0.0077108447, -0.009829209, -0.028392596, -0.0157867, -0.020531835, -0.007287172, -0.0038749767, 0.03887361, 0.011699887, -0.009555452, -0.027193276, 0.010702627, 0.031547327, -0.008245325, -0.0047483942, 0.030295864, 0.01572152, -0.020962026, 0.019723598, -0.16832194, 0.024351409, 0.01428755, -0.011197998, 0.013935576, 0.0022470953, 0.0088775745, 0.008949273, 0.002421453, 0.002887493, 0.009913944, -0.03407633, -0.018602494, -0.010819951, -0.008232288, -0.009020971, -0.020166826, 0.011184962, 0.01768997, -0.015356509, 0.008199698, 0.0015358139, 0.012827509, 0.010819951, 0.010663519, 0.011197998, -0.022774043, 0.017337995, -0.018954469, -0.010031268, 0.009822691, -0.022291707, 0.025668053, -0.0005238877, 0.024468733, 0.00669403, -0.008890611, 0.007841206, 0.0023595316, 0.017233705, 0.0031547328, 0.006345315, 0.0022829445, -0.0055142646, -0.021639902, 0.012227849, 0.0067526926, -0.0070329686, 0.022148311, -0.026958626, 0.00762611, -0.023099944, 0.0062638395, 0.021548651, 0.012436426, 0.038821466, -0.005752173, 0.006205177, 0.016933875, -0.005758691, -0.020857738, -0.00045422613, 0.02341281, -0.014848102, -0.011745513, -0.02497714, -0.010083413, 0.010937276, -0.016229928, 0.017337995, -0.017846402, -0.03840431, -0.00028455333, -0.032616287, 0.007952012, 0.013238145, -0.011901947, 0.010252882, -0.010122521, -0.0003434194, -0.02521179, 0.038560744, -0.033919897, 0.017937655, -0.01338806, 0.019866996, -0.009672776, 0.0143918395, 0.004706027, 0.009425091, 0.009412054, -0.020557908, -0.024846781, -0.022330815, 0.02411676, 0.015447762, 0.008010675, 0.022956548, -0.00043670888, 0.014613452, 0.0010217032, -0.008466938, 0.001668619, -0.00077116594, 0.027688647, 0.023803893, 0.011999717, 0.026072172, 0.009588041, 0.0011504346, -0.015069716, -0.010826469, 0.017155489, 0.02128793, 0.007723881, 0.005315464, -0.011921501, -0.020570943, 0.024938032, 0.015617231, 0.010181183, 0.0146916695, -0.011074156, 0.02737578, 0.0037152846, -0.007984603, -0.05026715, -0.009685812, 0.01638636, 0.008506047, -0.020948991, 0.03707463, -0.010129039, 0.04142868, 0.016425468, 0.04236728, 0.01458738, -0.028705461, 0.013361989, -0.0043638297, 0.017703004, -0.002951044, -0.00020572574, 0.0007463159, -0.004686473, 0.020610053, -0.01295787, -0.029722277, 0.02358228, -0.02111846, -0.0038358683, -0.0071307393, -0.033189874, -0.0011813953, 0.00286468, 0.008551672, 0.00066809944, -0.008499528, 0.010924241, -0.02467731, -0.004539817, 0.021196676, -0.024247121, 0.010904686, 0.028418668, -0.03397204, -0.00064691575, -0.0067526926, -0.017233705, -0.040881164, 0.028992256, -0.0096467035, -0.011810694, 0.021692047, -0.01115889, -0.00038436084, 0.0062931706, 0.016972983, -0.009607595, -0.004719063, -0.013361989, 0.015108824, 0.0058336486, -0.0063713873, -0.013596638, 0.002388863, -0.0011887281, -0.028496884, 0.0074696774, 0.0035425564, 0.03300737, 0.026802193, -0.014365767, 0.0054327887, 0.027193276, -0.009034008, -0.023738712, 0.021809371, -0.03493671, 0.029696204, -0.03000907, -0.010832988, -0.004565889, -0.0014901876, 0.012664557, 0.004526781, -0.028288307, -0.016255999, -0.017546572, -0.011347913, -0.0018299406, 0.0031612508, 0.0062116953, 0.018302664, 0.003078146, -0.054647274, 0.017259778, 0.023699604, 0.014378803, 0.00876025, -0.007573966, 0.0007613889, -0.00666144, 0.0055077462, -0.0020108162, -0.011217552, -0.038560744, -0.036761764, -0.084838845, 0.01788551, 0.018758928, -0.02638504, -0.0062410263, -0.01709031, 0.008290951, -0.022552429, 0.003353533, -0.004236728, -0.006224731, 0.018563386, 0.016738335, 0.0058532027, -0.012182223, 0.0052339886, 0.010031268, 0.0037511338, -0.005478415, 0.0012376134, 0.0037218025, -0.011224071, 0.004060741, -0.0005132959, 0.0043410165, 0.012912244, -0.012775364, 0.010787361, -0.022395996, -0.0076782545, 0.022800114, -0.042601928, 0.005585963, 0.03240771, -0.009399018, -0.02531608, -0.0038521634, 0.020062536, -0.001242502, 0.008121482, -0.03673569, -0.012618931, 0.021327037, -0.019945212, -2.3335104e-05, 0.017611753, -0.013596638, -0.010350652, -0.005745655, 3.1368083e-05, 0.0017631307, 0.022343852, -0.01608653, -0.021874553, -0.028184019, -0.007267618, 0.0039923014, -0.026398074, 0.0023253118, -0.0205188, 0.04552201, 0.01998432, 0.025498584, -0.0033665693, 0.02688041, -0.0023839741, -0.007443605, 0.021561686, -0.0011968757, -0.042758364, -0.011035047, -0.016738335, 0.010552712, 0.01332288, 0.022734934, -0.0062801344, -0.01299046, 0.005765209, -0.018211413, 0.035093144, 0.024729455, 0.0047027683, -0.051779334, 0.00065098953, 0.025850559, 0.0017256519, -0.020245042, 0.006687512, -0.017351031, 0.0034708579, -0.024872852, 0.00696127, -0.036109958, 0.00012394466, -0.0017859438, 0.025824487, 0.0044387872, -0.011563009, 0.024729455, 0.021639902, 0.013570566, 0.009001417, -0.006625591, -0.009040526, -0.034415267, 0.010989421, -0.02987871, -0.023634424, 0.0040020784, 0.02611128, 0.027949369, 0.020336295, 0.024142832, 0.0005809206, -0.00050677784, 0.011308805, -0.0040183733, -0.0024752268, -0.028470812, 0.027453998, -0.024142832, 0.0020368884, 0.0152913295, -0.0037380976, 0.012886171, 0.0075283395, 0.0043051676, -0.0076456647, 0.00021876182, -0.0018576423, -0.008688551, 0.02008861, -0.013505385, 0.0019765967, -0.023060836, -0.0034024185, -0.022800114, 0.01428755, -0.0046017384, 0.045287363, 0.015330438, -0.011211034, 0.011171926, 0.006987342, 0.0005947714, 0.015030608, 0.003614255, -0.009685812, -0.036109958, 0.0035979599, 0.006234508, -0.004070518, -0.02977442, -0.010546193, -0.028835822, -0.008688551, -0.018341772, -0.020075573, 0.0127232205, 0.022630645, 0.0022976103, 0.025107501, 0.005602258, -0.028444739, 0.007997639, -0.0032606511, -0.002341607, 0.00035727024, -0.04377518, -0.008740696, -0.023360666, -0.019945212, -0.02641111, -0.0065604104, 0.008662479, -0.011048083, -0.01035717, 0.025641982, -0.0005894755, -0.010774326, 0.026267713, -0.012495088, 0.0012367987, -0.00669403, 0.0065604104, -0.013231628, -0.002287833, -0.022057058], index=5, object='embedding')], model='text-embedding-ada-002-v2', object='list', usage=Usage(prompt_tokens=86, total_tokens=86))

and load the embeddings into a torch tensor at the device we defined earlier

embeddings = torch.tensor([
    r.embedding
    for r in response.data
]).to(device)
embeddings
tensor([[-0.0037, -0.0068,  0.0186,  ..., -0.0006, -0.0140, -0.0351],
        [-0.0089,  0.0201,  0.0002,  ...,  0.0086, -0.0182, -0.0043],
        [-0.0106, -0.0030, -0.0082,  ...,  0.0359, -0.0133, -0.0311],
        [-0.0140,  0.0072, -0.0037,  ..., -0.0261,  0.0062, -0.0386],
        [-0.0168,  0.0099, -0.0086,  ...,  0.0075, -0.0166, -0.0123],
        [-0.0117,  0.0066, -0.0059,  ..., -0.0132, -0.0023, -0.0221]],
       device='mps:0')

finally, we will use the vec2text package to perform embedding inversion

%%time

decoded = vec2text.invert_embeddings(
    embeddings=embeddings,
    corrector=corrector
)
decoded
CPU times: user 1.46 s, sys: 400 ms, total: 1.86 s
Wall time: 2.06 s
['How much text does the embedded text reveal about a private text?',
 'We investigate the problem of full embedding, by reconstructing the dense text in the context of an inverted embedding reconstruction.',
 'We try to generate a controlled, fixed-point vector that is reconstructed in such a way that the problem is close to generating reconstructed text: the space around the point is fixed.',
 'My phone number is 5555555555',
 'Daniel Daniel. email: frank_frank@icloud.com',
 'Anytown, USA 14 mile Main Street, Anytown, USA 14 mile Main Street, Anytown, USA']

Results

Let’s see the results of our experiment

df = pd.DataFrame({
    'text': text_samples,
    'decoded': decoded
})

df['bleu'] = df.apply(lambda r: 
    nltk.translate.bleu_score.sentence_bleu(
        references=[r.text],
        hypothesis=r.decoded
    )
,axis=1)

avg_blue = df['bleu'].mean()

df = df.style.background_gradient(subset=['bleu'])
df
  text decoded bleu
0 How much private information do text embeddings reveal about the original text? How much text does the embedded text reveal about a private text? 0.588642
1 We investigate the problem of embedding inversion, reconstructing the full text represented in dense text embeddings. We investigate the problem of full embedding, by reconstructing the dense text in the context of an inverted embedding reconstruction. 0.723020
2 We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We try to generate a controlled, fixed-point vector that is reconstructed in such a way that the problem is close to generating reconstructed text: the space around the point is fixed. 0.485054
3 My phone number is 555-555-5555 My phone number is 5555555555 0.824342
4 daniel.fat@icloud.com Daniel Daniel. email: frank_frank@icloud.com 0.334431
5 1234 Main Street, Anytown, USA Anytown, USA 14 mile Main Street, Anytown, USA 14 mile Main Street, Anytown, USA 0.320769 
f'Average BLEU score: {avg_blue:.2f}'
'Average BLEU score: 0.55'

Conclusion

In this notebook, we explored the capabilities of embedding inversion, a process of transforming dense text embeddings back into the original text. This approach is a form of controlled text generation, aiming to align generated text with specific points in the latent space represented by the embeddings. We used the vec2text python package to perform embedding inversion, which is a direct application of the research in the paper.

References