In-database Embedding Generation

Embedding generation is a process of transforming high-dimensional data into dense vectors of fixed size, which can be used for various machine learning tasks. PostgresML makes it easy to generate embeddings from text in your database using state-of-the-art models with the native function pgml.embed(model_name, text), leveraging the computational power of local GPUs.

Introduction

Different models have been trained on different types of text and with different algorithms. Each one has its own tradeoffs, generally latency vs quality, although recent progress in the LLMs.

Benefits of in-database processing

PostgresML cloud databases include GPU hardware to run state-of-the-art models for embedding generation within the database environment, among other ML/AI workloads. This contrasts with network calls, where data must be sent to an external service for processing. If you're running PostgresML on your own hardware it's important to configure it correctly, or choose an embedding model that will run efficiently on a CPU.

Reduced Latency: Local computation eliminates the need for network calls, significantly reducing latency.
Enhanced Security: Data remains within the database, enhancing security by minimizing exposure.
Cost-Effectiveness: Utilizing local hardware can be more cost-effective than relying on external services, especially for large-scale operations.

GPU accelerated models can compute embeddings in sub millisecond timeframes when batching, this means that even in-datacenter processing is orders of magnitude more expensive than in-database, in terms of latency and finances due to the networking overhead. Using a hosted service to generate embeddings outside-of your datacenter, is even less efficient, given the additional overhead of transport costs.

Model Selection

There are many excellent pre-trained open-weight models available for download from HuggingFace. PostgresML serverless instances run with the following models available w/ instant autoscaling:

Model	Parameters (M)	Strengths
intfloat/e5-small-v2	33.4	High quality, lowest latency
mixedbread-ai/mxbai-embed-large-v1	335	Higher quality, higher latency
Alibaba-NLP/gte-large-en-v1.5	434	Supports up to 8k token inputs

If you'd like to use a different model you can also provision dedicated resources for it. The Massive Text Embedding Benchmark is a helpful resource provided by HuggingFace that maintains up-to-date rankings on the latest models.

Creating Embeddings

You can generate embeddings using pgml.embed(model_name, text). For example:


                content_copy
            
SELECT pgml.embed('intfloat/e5-small-v2', 'This is some text to embed');

timer 12.029 ms


                content_copy
            
{-0.080910146,0.033980247,0.052564066,0.0020346553,-0.03936229,0.031479727,0.0685036,-0.06294509,-0.024574954,0.040237393,0.051508162,0.0038814095,-0.010645757,0.020144403,0.031223888,-0.04440482,0.020333821,0.07103317,-0.12705344,0.030591827,0.07019173,-0.036886554,-0.012233759,-0.07092232,0.0027690812,-0.0020539823,0.040779375,0.05908495,-0.026025668,-0.08242788,-0.018558107,-0.0094666025,0.059807047,-0.02525427,0.103207916,-0.068966456,-0.039847758,0.04071019,0.04450286,0.03424993,-0.06227554,-0.055733517,0.054585237,-0.060373828,-0.024653753,0.009867895,-0.041141387,-0.08721736,0.08264962,-0.0031608255,-0.012134463,-0.014921003,0.04267465,0.029093502,0.058714338,0.023871746,0.027041607,0.05843493,0.04142925,0.09514731,-0.030493727,0.07500542,-0.11280806,0.10281551,0.055736117,0.061823647,-0.020118464,0.014440284,-0.08269981,0.0040008957,-0.018531831,-0.008568512,-0.046970874,0.04578424,-0.039577056,0.08775033,-0.008210567,0.051924113,-0.04171466,-0.0367731,-0.01827072,0.0069318637,-0.047051124,0.033687923,0.0075546373,-0.037275027,0.043123465,-0.045893792,-0.036658753,-0.040635854,-0.03440536,0.0011549098,0.042740136,-0.025120102,-0.017873302,-0.039899718,0.031648446,0.0068402113,0.02402832,0.089285314,0.017456057,0.012008715,0.0076218387,-0.07197755,-0.038144454,-0.05969434,0.0389503,-0.0058245854,0.01937407,-0.018212182,-0.06195428,-0.038283527,-0.01753182,-0.023789542,0.07097847,0.04855445,-0.05200343,-0.009433737,-0.010195946,0.00442146,0.043388885,-0.013206756,0.03384104,0.0052567925,0.10585855,-0.08633147,0.05733634,0.046828035,0.111744046,-0.016215837,0.031619936,-0.0007159129,-0.0209652,-0.015532438,-0.06690792,-0.0091873575,-0.044681326,-0.007757966,0.053561073,-0.011261849,-0.03140146,-0.050118096,-0.031356297,-0.124189764,0.024152948,0.02993825,-0.07240996,0.01793813,-0.070896275,-0.024419364,-0.040071633,-0.026535412,0.027830372,0.021783136,-0.0075028464,0.014013486,-0.005176842,0.044899847,-0.068266265,-0.024272943,-0.104513876,-0.007814491,0.06390731,0.10318874,0.08249727,-0.092428714,0.0062611965,-0.0115522025,0.056004044,-0.043695573,-0.0010207174,0.013102924,-0.0035022667,0.0025919478,0.12973104,-0.053112745,-0.008374208,-0.022599943,0.04597443,-0.074845895,0.07259128,-0.062168732,-0.03033916,0.03646452,0.033044446,-0.040221635,-0.060735658,-0.040255345,0.013989559,-0.026528435,-0.059659433,-0.0010745272,-0.02860176,0.073617734,0.009127505,0.012357427,-0.024373775,-0.07039051,-0.038225688,-0.07232986,0.06928063,0.06729482,-0.07500053,0.0036577163,-0.03904865,0.09585222,0.035453793,-0.0061846063,-0.05000263,-0.050227694,-0.022932036,-0.0073578595,-0.034768302,-0.038604897,-0.01470962,-0.04274356,-0.01689811,0.04931222,0.010990732,0.019879386,0.01243605,-0.07632878,-0.070137314,-0.15282577,-0.020428825,-0.030160243,-0.0050396603,0.007732285,-0.032149784,-0.015778365,0.07480648,0.017192233,0.024550207,0.06951421,-0.014848112,-0.05396024,-0.03223639,0.04666939,0.012844642,-0.05892448,-0.030294335,0.06794056,-0.063875966,-0.046530016,-0.07084713,-0.031829637,-0.047059055,0.08617301,-0.05032479,0.118310556,0.04755146,-0.028393123,-0.024320556,0.030537084,0.020449162,0.05665035,-0.075432904,0.07822404,-0.07196871,0.010495469,0.05382172,-0.0016319404,-0.087258086,0.0930253,-0.01846083,0.0033103244,-0.08890738,0.071200974,-0.03997286,-0.005042026,0.011910354,-0.025650134,0.054577664,-0.0014927471,-0.047521923,0.049124297,0.006342861,-0.089150384,-0.0073342607,-0.07849969,0.0010329112,-0.038727123,0.016429648,-0.086470395,-4.8742084e-05,0.060051307,0.0033317064,0.006863758,0.0446841,-0.031092882,0.017449407,-0.07479843,-0.058406148,-0.012044445,0.08927765,-0.04008159,0.05227031,0.021864118,0.054245688,0.027357962,0.02569578,-0.06151034,-0.05588746,-0.034790445,-0.020313034,0.03713666,0.025836824,0.039398894,0.02515884,-0.008512022,-0.014856683,0.037740804,-0.06471344,0.029907772,0.0077477624,0.061302595,0.037709966,-0.032406874,-0.049870085,-0.15800017,-0.014624413,0.018514019,-0.010369406,-0.022790398,0.009587365,0.03241724,-0.02795245,-0.05280684,-0.031362813,0.047515675,0.009669598,0.09689132,-0.038499177,-0.019239947,0.06885492,0.08843166,-0.027636368,-0.058589518,-0.11492329,0.036349587,0.03926196,0.16907486,0.036197387,-0.0128475325,0.05160944,0.0034505632,0.016367715,0.068978526,0.0676247,0.0064224014,-0.06316567,0.11720159,0.005348484,0.05403974,0.061581556,-0.027833184,0.05563025,0.03337182,-0.030032963,0.06838953,0.08052612,-0.01996433,0.006692282,0.11277913,0.03004468,-0.063005574,-0.024108425,-0.03547973,0.0060482216,-0.0032331524,-0.038302638,0.083412275,0.07387719,0.052097928,-0.037775334,-0.05458932,0.0004270608,-0.034030076,-0.07965879,0.012511749,-0.028165875,0.03768439,0.00082042674,0.053660177}

A database typically holds the text data used to generate the embeddings in a table. We'll use documents as an example.


                content_copy
            
CREATE TABLE documents (
   id SERIAL PRIMARY KEY,
   body TEXT
);

Inserting some example data:


                content_copy
            
INSERT INTO documents (body)
VALUES
    ('Example text data'),
    ('Another example document'),
    ('Some other thing');

Passing the data from the table to the embedding function:


                content_copy
            
SELECT id, pgml.embed('intfloat/e5-small-v2', body)
FROM documents;

timer 50.001 ms


                content_copy
            
  id |                                                                                                                                                                                                                                                                                                                                                                                                     embed
---+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1 | {-0.09234577,0.037487056,-0.03421769,-0.033738457,-0.042548284,-0.0015319627,0.042109113,0.011365055,-0.018372666,0.020417988,0.061961487,-0.022707041,0.015810987,0.03675479,0.001995532,-0.04197657,-0.034883354,0.07871886,-0.11676137,0.06141681,0.08321331,-0.03457781,-0.013248807,-0.05802344,-0.039144825,-0.015038275,0.020686107,0.08593334,-0.041029375,-0.13210341,-0.034079146,0.016687978,0.06363906,-0.05279167,0.10102262,-0.048170853,-0.014849669,0.03523273,0.024248678,0.031341534,-0.021447029,-0.05781338,0.039722513,-0.058294114,-0.035174508,-0.056844078,-0.051775914,-0.05822031,0.083022244,0.027178412,0.0032413877,0.023898097,0.023951318,0.0565093,0.036267336,0.049430914,0.027110789,0.05017207,0.058326595,0.040568575,0.014855128,0.06272174,-0.12961388,0.0998898,0.014964503,0.07735804,-0.028795758,0.026889611,-0.0613238,-0.004798127,0.009027658,0.046634953,-0.034936648,0.076499216,-0.03855506,0.08894715,-0.0019889707,0.07027481,-0.04624302,-0.048422314,-0.02444203,-0.0442959,-0.028878363,0.04586853,-0.004158767,-0.0027680802,0.029728336,-0.06130052,-0.028088963,-0.050658133,-0.024370935,-0.0030779864,0.018137587,-0.029853988,-0.06877675,-0.001238518,0.025249483,-0.0045243553,0.07250941,0.12831028,0.0077543575,0.012130527,-0.0006014347,-0.027807593,-0.011226617,-0.04837827,0.0376276,-0.058811083,0.020967057,-0.021439878,-0.0634577,-0.029189702,-0.040197153,-0.01993339,0.0899751,-0.014370172,0.0021994617,-0.0759979,-0.010541287,0.034424484,0.030067233,0.016858222,0.015223163,0.021410512,0.072372325,-0.06270684,0.09666927,0.07237114,0.09372637,-0.027058149,0.06319879,-0.03626834,-0.03539027,0.010406426,-0.08829164,-0.020550422,-0.043701466,-0.018676292,0.038060706,-0.0058152666,-0.04057362,-0.06266247,-0.026675962,-0.07610313,-0.023740835,0.06968648,-0.076157875,0.05129882,-0.053703927,-0.04906172,-0.014506706,-0.033226766,0.04197027,0.009892002,-0.019509513,0.020975547,0.015931072,0.044290986,-0.048697367,-0.022310019,-0.088599496,-0.0371257,0.037382104,0.14381507,0.07789086,-0.10580675,0.0255245,0.014246269,0.01157928,-0.069586724,0.023313843,0.02494169,-0.014511085,-0.017566541,0.0865948,-0.012115137,0.024397936,-0.049067125,0.03300015,-0.058626212,0.034921415,-0.04132337,-0.025009211,0.057668354,0.016189015,-0.04954466,-0.036778226,-0.046015732,-0.041587763,-0.03449501,-0.033505566,0.019262834,-0.018552447,0.019556912,0.01612039,0.0026575527,-0.05330489,-0.06894643,-0.04592849,-0.08485257,0.12714563,0.026810834,-0.053618323,-0.029470881,-0.04381535,0.055211045,-0.0111715235,-0.004484313,-0.02654065,-0.022317547,-0.027823675,0.0135190515,0.001530742,-0.04323672,-0.028350104,-0.07154715,-0.0024147208,0.031836234,0.03476004,0.033611998,0.038179073,-0.087631755,-0.048408568,-0.11773682,-0.019127818,0.013682835,-0.02015055,0.01888005,-0.03280704,0.0076310635,0.074330166,-0.031277154,0.056628436,0.119448215,-0.0012235055,-0.009727585,-0.05459528,0.04298459,0.054554865,-0.027898816,0.0040641865,0.08585007,-0.053415824,-0.030528797,-0.08231634,-0.069264784,-0.08337459,0.049254872,-0.021684796,0.12479715,0.053940497,-0.038884085,-0.032209005,0.035795107,0.0054665194,0.0085438965,-0.039386917,0.083624765,-0.056901276,0.022051739,0.06955752,-0.0008329906,-0.07959222,0.075660035,-0.017008293,0.015329365,-0.07439257,0.057193674,-0.06564091,0.0007063081,-0.015799401,-0.008529507,0.027204275,0.0076780985,-0.018589584,0.065267086,-0.02026929,-0.0559547,-0.035843417,-0.07237942,0.028072618,-0.048903402,-0.027478782,-0.084877744,-0.040812787,0.026713751,0.016210195,-0.039116003,0.03572044,-0.014964189,0.026315138,-0.08638934,-0.04198059,-0.02164005,0.09299754,-0.047685668,0.061317034,0.035914674,0.03533252,0.0287274,-0.033809293,-0.046841178,-0.042211317,-0.02567011,-0.048029255,0.039492987,0.04906847,0.030969618,0.0066106897,0.025528666,-0.008357054,0.04791732,-0.070402496,0.053391967,-0.06309544,0.06575766,0.06522203,0.060434356,-0.047547556,-0.13597175,-0.048658505,0.009734684,-0.016258504,-0.034227647,0.05382081,0.001330341,0.011890187,-0.047945525,-0.031132223,0.0010349775,0.030007072,0.12059559,-0.060273632,-0.010099646,0.055261053,0.053757478,-0.045518342,-0.041972063,-0.08315036,0.049884394,0.037543204,0.17598632,-0.0027433096,0.015989233,0.017486975,0.0059954696,-0.022668751,0.05677827,0.029728843,0.0011321013,-0.051546678,0.1113402,0.017779723,0.050953783,0.10342974,0.04067395,0.054890294,0.017487328,-0.020321153,0.062171113,0.07234749,-0.06777497,-0.03888628,0.08744684,0.032227095,-0.04398878,-0.049698275,-0.0018518695,-0.015967874,-0.0415869,-0.022655524,0.03596353,0.07130526,0.056296617,-0.06720573,-0.092787154,0.021057911,0.015628621,-0.04396636,-0.0063872878,-0.0127499355,0.01633339,-0.0006204544,0.0438727}
 2 | {-0.11384405,0.067140445,0.004428383,-0.019213142,0.011713443,0.009808596,0.06439777,-0.014959955,-0.03600561,0.01949383,0.04094742,0.030407589,-0.026018979,0.044171993,0.022412317,-0.057937913,-0.05182386,0.07793179,-0.109105654,0.057499174,0.102279164,-0.04705679,0.0010215766,-0.052305017,-0.0064890077,-0.019298203,0.0027092565,0.07363092,-0.010116459,-0.12196041,-0.025577176,0.010314696,0.031369787,-0.020949671,0.08722754,-0.051809352,0.0007810379,0.07672705,-0.008455481,0.06511949,-0.021327827,-0.060510863,0.044916406,-0.08674781,-0.047401372,-0.01868107,-0.075262256,-0.055392392,0.072947465,-0.01151735,-0.0072187134,0.015544381,0.039965566,0.020232335,0.04894269,0.04900096,0.05358905,0.032501124,0.053288646,0.07584814,0.031957388,0.05976136,-0.12726106,0.103460334,0.06346268,0.06554993,-0.045167506,0.012330433,-0.062929176,0.043507233,-0.008544882,0.027812833,-0.040016085,0.055822216,-0.03835489,0.040096387,0.018063055,0.060356017,-0.0726533,-0.0671456,-0.05047295,-0.042710193,-0.042777598,-0.006822609,0.012524907,-0.032105528,0.026691807,-0.05756205,0.015424967,-0.04767447,-0.036748573,-0.02527533,0.025934244,-0.033328723,-4.1858173e-05,-0.027706677,0.047805857,0.00018475522,0.050902035,0.1352519,0.005388455,0.029921843,-0.02537518,-0.058101207,-0.021984883,-0.059336115,0.03498545,-0.052446626,0.022411253,0.0060822135,-0.068493545,-0.013820616,-0.03522277,-0.018971028,0.07487064,-0.0009035772,-0.009381329,-0.04850395,0.001105027,0.016467793,0.0268643,0.0013964645,0.043346133,-0.009041368,0.07489963,-0.07887815,0.068340026,0.03767777,0.11665796,-0.025433592,0.062018104,-0.030672694,-0.012993033,0.0068405713,-0.03688894,-0.022034604,-0.040981747,-0.033101898,0.071058825,-0.0017327801,-0.021141728,-0.07144207,-0.02906128,-0.095396295,0.006055787,0.08500532,-0.031142898,0.055712428,-0.041926548,-0.042101618,-0.013311086,-0.046836447,0.023902802,0.031264246,-0.012085872,0.042904463,0.011645057,0.049069524,-0.0039288886,-0.014362478,-0.06809574,-0.038734697,0.028410498,0.12843607,0.090781115,-0.119838186,0.016676102,0.0009924435,0.0314442,-0.040607806,0.0020882064,0.044765383,0.01829387,-0.05677682,0.08415222,-0.06399008,-0.010945022,-0.024140757,0.046428833,-0.0651499,0.041250102,-0.06294866,-0.032783676,0.047456875,0.034612734,-0.021892011,-0.050926965,-0.06388983,-0.031164767,0.053277884,-0.069394015,0.03465082,-0.0410735,0.03736871,0.010950864,0.01830701,-0.070063934,-0.06988279,-0.03560967,-0.05519299,0.07882521,0.05533408,-0.02321644,0.007326457,-0.05126765,0.045479607,0.01830127,-0.037239183,-0.08015762,-0.056017533,-0.07647084,-0.0065865014,-0.027235825,-0.039984804,-0.0156225115,-0.014561295,0.024489071,0.009097713,0.04265267,-0.003169223,0.010329996,-0.078917705,-0.026417341,-0.13925064,-0.009786513,-0.037679326,-0.023494951,0.016230932,-0.010068113,0.008919443,0.05672694,-0.0647096,0.0074613485,0.0856074,-0.0072963624,-0.04508945,-0.027654354,0.031864826,0.046863783,-0.032239847,-0.024967564,0.065593235,-0.05142123,-0.011477745,-0.083396286,-0.036403924,-0.030264381,0.060208946,-0.037968345,0.13118903,0.055968005,-0.02204113,-0.00871512,0.06265703,0.024767108,0.06307163,-0.093918525,0.06388834,-0.027308429,0.028177679,0.046643235,-0.008643308,-0.08599351,0.08742052,-0.0045658057,0.009925819,-0.061982065,0.06666853,-0.085638665,-0.008682048,0.016528588,-0.015443429,0.040419903,0.0059123226,-0.04848474,0.026133329,-0.042095724,-0.06860905,-0.033551272,-0.06492134,0.019667841,-0.04917464,-0.0096588,-0.10072659,-0.07769663,0.03221359,0.019174514,0.039727442,0.025392585,-0.016384942,0.0024048705,-0.09175566,-0.03225071,0.0066428655,0.10759633,-0.04011207,0.031578932,0.06299788,0.061487168,0.048043367,-0.0047893273,-0.054848563,-0.06647676,-0.027905045,-0.055799212,0.028914401,0.04013868,0.050728165,-0.0063177645,-0.018899892,0.008193828,0.025991635,-0.08009935,0.044058595,-0.046858713,0.072079815,0.046664152,0.019002488,-0.018447064,-0.15560018,-0.050175466,0.001016439,-0.0035773942,-0.025972001,0.047064543,0.01866733,0.0049167247,-0.052880444,-0.029235922,-0.024581103,0.040634423,0.095990844,-0.019483034,-0.02325509,0.056078408,0.09241045,-0.03079215,-0.023518562,-0.08394134,0.03326668,0.008070111,0.14776507,0.030338759,-0.01846056,0.009517991,0.0034727904,0.007246884,0.015436005,0.058226254,-0.037932027,-0.04309255,0.09766471,0.014914252,0.03149386,0.10146584,0.009303289,0.05649276,0.04743103,-0.016993523,0.054828145,0.033858124,-0.059207607,-0.027288152,0.09254907,0.07817234,-0.047911037,-0.023988279,-0.067968085,-0.03140125,-0.02434741,-0.017226815,0.050405838,0.048384074,0.10386314,-0.05366119,-0.048218876,0.022471255,-0.04470827,-0.055776954,0.0146418335,-0.03505756,0.041757654,0.0076765255,0.0637766}
 3 | {-0.06530473,0.043326367,0.027487691,-0.012605501,-0.003679171,0.0068843057,0.093755856,-0.018192727,-0.038994554,0.060702052,0.047350235,0.0015797003,-0.026038624,0.029946782,0.053223953,-0.009188536,-0.012273773,0.07512682,-0.1220027,0.024623549,0.040207546,-0.061494265,-0.0016338134,-0.096063755,-0.020626824,-0.0008177105,0.025736991,0.08205663,-0.064413406,-0.10329614,-0.050153203,0.022038238,-0.011629073,-0.03142779,0.09684598,-0.045188677,-0.032773193,0.041901052,0.032470446,0.06218501,0.00056252955,-0.03571358,0.030095506,-0.09239761,-0.020187493,-0.00932361,-0.08373726,-0.053929392,0.09724756,-0.032078817,0.02658544,0.009965162,0.07477913,0.05487153,0.023828406,0.06263976,0.06882497,0.08249143,0.062069558,0.08915651,-0.005154778,0.056259956,-0.13729677,0.08404741,0.07149277,0.04482675,-0.058625933,0.0034976404,-0.030747578,0.004520399,0.0007449915,9.660358e-05,-0.022526976,0.11449101,-0.043607008,0.026769284,0.021050733,0.05854427,-0.042627476,-0.022924222,-0.059794623,-0.037738875,-0.018500011,0.017315088,-0.00020744087,-0.0016206327,0.013337528,-0.022439854,-0.0042932644,-0.04706647,-0.06771751,-0.040391076,0.0638978,-0.031776994,0.011536817,-0.04593729,0.08626801,0.0016808647,-0.0046028513,0.13702579,0.02293593,0.043189116,-0.0073873955,-0.06097065,-0.019305069,-0.025651531,0.043129053,-0.033460874,0.03261353,-0.022361644,-0.07769732,-0.021210406,-0.020294553,-0.044899672,0.083500296,0.038056396,-0.052046232,-0.03215008,-0.028185,0.041909587,0.016012225,-0.0058236965,0.021344814,-0.037620485,0.07454872,-0.03517924,0.086520284,0.096695796,0.0937938,-0.04190071,0.072271764,-0.07022541,0.01583733,-0.0017275782,-0.05280332,-0.005904967,-0.046241984,-0.024421731,0.09988276,-0.0077029592,-0.04107849,-0.091607556,0.033811443,-0.1323201,-0.015927043,0.011014193,-0.039773338,0.033963792,-0.053305525,-0.005038948,-0.024107914,-0.0079898145,0.039604105,0.009226985,0.0010978039,-0.015565131,-0.0002796709,0.037623808,-0.059376597,0.015390821,-0.07600872,-0.008280972,0.023050148,0.0777234,0.061332665,-0.13979945,-0.009342198,0.012803744,0.049805813,-0.03578894,-0.05038778,0.048912454,0.032017626,0.015345221,0.10369494,-0.048897773,-0.054201737,-0.015793057,0.08130064,-0.064783126,0.074246705,-0.06964914,-0.025839275,0.030869238,0.06357789,-0.028754702,-0.02960897,-0.04956488,0.030501548,0.005857936,-0.023547728,0.03717584,0.0024309678,0.066338174,-0.009775384,-0.030799516,-0.028462514,-0.058787093,-0.051071096,-0.048674088,0.011397957,0.07817651,-0.03227047,0.027149512,-0.0030777291,0.061677814,0.0025318298,-0.027110869,-0.0691719,-0.033963803,-0.0648151,-0.033951994,-0.0478505,0.0016642202,-0.019602248,-0.030472266,0.015889537,-0.0009066139,0.032841947,0.021004336,-0.029254122,-0.09597239,-0.04359093,-0.15422617,-0.016366383,-0.059343938,-0.064871244,0.07659653,0.023196936,-0.021893008,0.080793895,-0.05248758,0.018764181,0.0008353451,-0.03318359,-0.04830206,-0.05518034,0.038481984,0.06544077,0.019498836,-0.054670736,0.040052623,-0.028875519,-0.047129385,-0.03614192,-0.012638911,-0.0042204396,0.013685266,-0.047130045,0.11024768,0.07135732,-0.017937008,-0.040911496,0.09008783,0.039298594,0.042975742,-0.08974752,0.08711358,-0.021977019,0.051495675,0.0140351625,-0.053809136,-0.08241595,0.04982693,-0.020355707,0.017629888,-0.039196398,0.08688628,-0.051167585,-0.029257154,0.009161573,-0.0021740724,0.027258197,0.015352816,-0.07426982,0.022452697,-0.041628033,-0.023250584,-0.051996145,-0.031867135,-0.01930267,-0.05257186,0.032619886,-0.08220233,-0.017010445,0.038414452,-0.02268424,0.007727591,0.0064041745,-0.024256933,0.0028989788,-0.06191567,-0.020444075,-0.010515549,0.08980986,-0.020033991,0.009208651,0.044014987,0.067944355,0.07915397,0.019362122,-0.010731527,-0.057449125,-0.007854527,-0.067998596,0.036500365,0.037355963,-0.0011789168,0.030410502,-0.012768641,-0.03281059,0.026916556,-0.052477527,0.042145997,-0.023683913,0.099338256,0.035008017,-0.029086927,-0.032222193,-0.14743629,-0.04350868,0.030494612,-0.013000542,0.021753347,0.023393912,0.021320568,0.0031570331,-0.06008047,-0.031103736,0.030275675,0.015258714,0.09004704,0.0033432578,-0.0045539658,0.06602429,0.072156474,-0.0613405,-0.047462273,-0.057639644,-0.008026253,0.03090332,0.12396069,0.04592149,-0.053269017,0.034282286,-0.0045666047,-0.026025562,0.004598449,0.04304216,-0.02252559,-0.040372007,0.08094969,-0.021883471,0.05903653,0.10130699,0.001840184,0.06142003,0.004450253,-0.023686321,0.014760433,0.07669066,-0.08392746,-0.028447477,0.08995419,0.028487092,-0.047503598,-0.026627144,-0.0475691,-0.069141485,-0.039571274,-0.054866526,0.04417342,0.08155949,0.065555565,-0.053984754,-0.04142323,-0.023902748,0.0066344747,-0.065118864,0.02183451,-0.06479133,0.010425607,-0.010283142,0.0940532}

We can store embeddings in the database as well. Here's an example of creating a temporary table to hold all the embeddings during the current transaction.


                content_copy
            
CREATE TEMPORARY TABLE embeddings AS
SELECT id AS document_id,
       pgml.embed('intfloat/e5-small-v2', body)
FROM documents;

timer 54.123 ms


                content_copy
            
SELECT 3

Another way would be to generated and store the embeddings any time a document is updated:


                content_copy
            
CREATE TABLE documents_with_embeddings (
id SERIAL PRIMARY KEY,
body TEXT,
embedding FLOAT[] GENERATED ALWAYS AS (pgml.normalize_l2(pgml.embed('intfloat/e5-small-v2', body))) STORED
);


                content_copy
            
INSERT INTO documents_with_embeddings (body)
VALUES -- embedding vectors are automatically generated
    ('Example text data'),
    ('Another example document'),
    ('Some other thing');

timer 46.823


                content_copy
            
INSERT 0 3

You could also use a Common Table Expression to generate an embedding on the fly and then reference it later in the SQL statement. For example, to generate a search embedding, and compare it to all existing embeddings in a table to find the nearest neighbors:


                content_copy
            
WITH query AS (
    SELECT pgml.embed('intfloat/e5-small-v2', 'An example search query') AS embedding
)
SELECT id, pgml.distance_l2(query.embedding, documents_with_embeddings.embedding)
FROM documents_with_embeddings, query
ORDER BY distance_l2;

timer 25.688 ms


                content_copy
            
 id |     distance_l2
----+---------------------
  1 | 0.45335962377530326
  2 | 0.49441662560530825
  3 |   0.632445005046323

Batching

PostgresML supports batching embeddings. It turns out, a lot of the cost of generating an embedding is streaming the model weights for each layer from memory to the processors, rather than performing the actual calculations. By batching embeddings, we can reuse the weights for each layer on multiple inputs, before loading the next layer and continuing, which amortizes the RAM latency across all embeddings.


                content_copy
            
SELECT pgml.embed('intfloat/e5-small-v2', array_agg(body)) AS embedding
FROM documents;

timer 21.204 ms


                content_copy
            
  id |                                                                                                                                                                                                                                                                                                                                                                                                     embed
---+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1 | {-0.09234577,0.037487056,-0.03421769,-0.033738457,-0.042548284,-0.0015319627,0.042109113,0.011365055,-0.018372666,0.020417988,0.061961487,-0.022707041,0.015810987,0.03675479,0.001995532,-0.04197657,-0.034883354,0.07871886,-0.11676137,0.06141681,0.08321331,-0.03457781,-0.013248807,-0.05802344,-0.039144825,-0.015038275,0.020686107,0.08593334,-0.041029375,-0.13210341,-0.034079146,0.016687978,0.06363906,-0.05279167,0.10102262,-0.048170853,-0.014849669,0.03523273,0.024248678,0.031341534,-0.021447029,-0.05781338,0.039722513,-0.058294114,-0.035174508,-0.056844078,-0.051775914,-0.05822031,0.083022244,0.027178412,0.0032413877,0.023898097,0.023951318,0.0565093,0.036267336,0.049430914,0.027110789,0.05017207,0.058326595,0.040568575,0.014855128,0.06272174,-0.12961388,0.0998898,0.014964503,0.07735804,-0.028795758,0.026889611,-0.0613238,-0.004798127,0.009027658,0.046634953,-0.034936648,0.076499216,-0.03855506,0.08894715,-0.0019889707,0.07027481,-0.04624302,-0.048422314,-0.02444203,-0.0442959,-0.028878363,0.04586853,-0.004158767,-0.0027680802,0.029728336,-0.06130052,-0.028088963,-0.050658133,-0.024370935,-0.0030779864,0.018137587,-0.029853988,-0.06877675,-0.001238518,0.025249483,-0.0045243553,0.07250941,0.12831028,0.0077543575,0.012130527,-0.0006014347,-0.027807593,-0.011226617,-0.04837827,0.0376276,-0.058811083,0.020967057,-0.021439878,-0.0634577,-0.029189702,-0.040197153,-0.01993339,0.0899751,-0.014370172,0.0021994617,-0.0759979,-0.010541287,0.034424484,0.030067233,0.016858222,0.015223163,0.021410512,0.072372325,-0.06270684,0.09666927,0.07237114,0.09372637,-0.027058149,0.06319879,-0.03626834,-0.03539027,0.010406426,-0.08829164,-0.020550422,-0.043701466,-0.018676292,0.038060706,-0.0058152666,-0.04057362,-0.06266247,-0.026675962,-0.07610313,-0.023740835,0.06968648,-0.076157875,0.05129882,-0.053703927,-0.04906172,-0.014506706,-0.033226766,0.04197027,0.009892002,-0.019509513,0.020975547,0.015931072,0.044290986,-0.048697367,-0.022310019,-0.088599496,-0.0371257,0.037382104,0.14381507,0.07789086,-0.10580675,0.0255245,0.014246269,0.01157928,-0.069586724,0.023313843,0.02494169,-0.014511085,-0.017566541,0.0865948,-0.012115137,0.024397936,-0.049067125,0.03300015,-0.058626212,0.034921415,-0.04132337,-0.025009211,0.057668354,0.016189015,-0.04954466,-0.036778226,-0.046015732,-0.041587763,-0.03449501,-0.033505566,0.019262834,-0.018552447,0.019556912,0.01612039,0.0026575527,-0.05330489,-0.06894643,-0.04592849,-0.08485257,0.12714563,0.026810834,-0.053618323,-0.029470881,-0.04381535,0.055211045,-0.0111715235,-0.004484313,-0.02654065,-0.022317547,-0.027823675,0.0135190515,0.001530742,-0.04323672,-0.028350104,-0.07154715,-0.0024147208,0.031836234,0.03476004,0.033611998,0.038179073,-0.087631755,-0.048408568,-0.11773682,-0.019127818,0.013682835,-0.02015055,0.01888005,-0.03280704,0.0076310635,0.074330166,-0.031277154,0.056628436,0.119448215,-0.0012235055,-0.009727585,-0.05459528,0.04298459,0.054554865,-0.027898816,0.0040641865,0.08585007,-0.053415824,-0.030528797,-0.08231634,-0.069264784,-0.08337459,0.049254872,-0.021684796,0.12479715,0.053940497,-0.038884085,-0.032209005,0.035795107,0.0054665194,0.0085438965,-0.039386917,0.083624765,-0.056901276,0.022051739,0.06955752,-0.0008329906,-0.07959222,0.075660035,-0.017008293,0.015329365,-0.07439257,0.057193674,-0.06564091,0.0007063081,-0.015799401,-0.008529507,0.027204275,0.0076780985,-0.018589584,0.065267086,-0.02026929,-0.0559547,-0.035843417,-0.07237942,0.028072618,-0.048903402,-0.027478782,-0.084877744,-0.040812787,0.026713751,0.016210195,-0.039116003,0.03572044,-0.014964189,0.026315138,-0.08638934,-0.04198059,-0.02164005,0.09299754,-0.047685668,0.061317034,0.035914674,0.03533252,0.0287274,-0.033809293,-0.046841178,-0.042211317,-0.02567011,-0.048029255,0.039492987,0.04906847,0.030969618,0.0066106897,0.025528666,-0.008357054,0.04791732,-0.070402496,0.053391967,-0.06309544,0.06575766,0.06522203,0.060434356,-0.047547556,-0.13597175,-0.048658505,0.009734684,-0.016258504,-0.034227647,0.05382081,0.001330341,0.011890187,-0.047945525,-0.031132223,0.0010349775,0.030007072,0.12059559,-0.060273632,-0.010099646,0.055261053,0.053757478,-0.045518342,-0.041972063,-0.08315036,0.049884394,0.037543204,0.17598632,-0.0027433096,0.015989233,0.017486975,0.0059954696,-0.022668751,0.05677827,0.029728843,0.0011321013,-0.051546678,0.1113402,0.017779723,0.050953783,0.10342974,0.04067395,0.054890294,0.017487328,-0.020321153,0.062171113,0.07234749,-0.06777497,-0.03888628,0.08744684,0.032227095,-0.04398878,-0.049698275,-0.0018518695,-0.015967874,-0.0415869,-0.022655524,0.03596353,0.07130526,0.056296617,-0.06720573,-0.092787154,0.021057911,0.015628621,-0.04396636,-0.0063872878,-0.0127499355,0.01633339,-0.0006204544,0.0438727}
 2 | {-0.11384405,0.067140445,0.004428383,-0.019213142,0.011713443,0.009808596,0.06439777,-0.014959955,-0.03600561,0.01949383,0.04094742,0.030407589,-0.026018979,0.044171993,0.022412317,-0.057937913,-0.05182386,0.07793179,-0.109105654,0.057499174,0.102279164,-0.04705679,0.0010215766,-0.052305017,-0.0064890077,-0.019298203,0.0027092565,0.07363092,-0.010116459,-0.12196041,-0.025577176,0.010314696,0.031369787,-0.020949671,0.08722754,-0.051809352,0.0007810379,0.07672705,-0.008455481,0.06511949,-0.021327827,-0.060510863,0.044916406,-0.08674781,-0.047401372,-0.01868107,-0.075262256,-0.055392392,0.072947465,-0.01151735,-0.0072187134,0.015544381,0.039965566,0.020232335,0.04894269,0.04900096,0.05358905,0.032501124,0.053288646,0.07584814,0.031957388,0.05976136,-0.12726106,0.103460334,0.06346268,0.06554993,-0.045167506,0.012330433,-0.062929176,0.043507233,-0.008544882,0.027812833,-0.040016085,0.055822216,-0.03835489,0.040096387,0.018063055,0.060356017,-0.0726533,-0.0671456,-0.05047295,-0.042710193,-0.042777598,-0.006822609,0.012524907,-0.032105528,0.026691807,-0.05756205,0.015424967,-0.04767447,-0.036748573,-0.02527533,0.025934244,-0.033328723,-4.1858173e-05,-0.027706677,0.047805857,0.00018475522,0.050902035,0.1352519,0.005388455,0.029921843,-0.02537518,-0.058101207,-0.021984883,-0.059336115,0.03498545,-0.052446626,0.022411253,0.0060822135,-0.068493545,-0.013820616,-0.03522277,-0.018971028,0.07487064,-0.0009035772,-0.009381329,-0.04850395,0.001105027,0.016467793,0.0268643,0.0013964645,0.043346133,-0.009041368,0.07489963,-0.07887815,0.068340026,0.03767777,0.11665796,-0.025433592,0.062018104,-0.030672694,-0.012993033,0.0068405713,-0.03688894,-0.022034604,-0.040981747,-0.033101898,0.071058825,-0.0017327801,-0.021141728,-0.07144207,-0.02906128,-0.095396295,0.006055787,0.08500532,-0.031142898,0.055712428,-0.041926548,-0.042101618,-0.013311086,-0.046836447,0.023902802,0.031264246,-0.012085872,0.042904463,0.011645057,0.049069524,-0.0039288886,-0.014362478,-0.06809574,-0.038734697,0.028410498,0.12843607,0.090781115,-0.119838186,0.016676102,0.0009924435,0.0314442,-0.040607806,0.0020882064,0.044765383,0.01829387,-0.05677682,0.08415222,-0.06399008,-0.010945022,-0.024140757,0.046428833,-0.0651499,0.041250102,-0.06294866,-0.032783676,0.047456875,0.034612734,-0.021892011,-0.050926965,-0.06388983,-0.031164767,0.053277884,-0.069394015,0.03465082,-0.0410735,0.03736871,0.010950864,0.01830701,-0.070063934,-0.06988279,-0.03560967,-0.05519299,0.07882521,0.05533408,-0.02321644,0.007326457,-0.05126765,0.045479607,0.01830127,-0.037239183,-0.08015762,-0.056017533,-0.07647084,-0.0065865014,-0.027235825,-0.039984804,-0.0156225115,-0.014561295,0.024489071,0.009097713,0.04265267,-0.003169223,0.010329996,-0.078917705,-0.026417341,-0.13925064,-0.009786513,-0.037679326,-0.023494951,0.016230932,-0.010068113,0.008919443,0.05672694,-0.0647096,0.0074613485,0.0856074,-0.0072963624,-0.04508945,-0.027654354,0.031864826,0.046863783,-0.032239847,-0.024967564,0.065593235,-0.05142123,-0.011477745,-0.083396286,-0.036403924,-0.030264381,0.060208946,-0.037968345,0.13118903,0.055968005,-0.02204113,-0.00871512,0.06265703,0.024767108,0.06307163,-0.093918525,0.06388834,-0.027308429,0.028177679,0.046643235,-0.008643308,-0.08599351,0.08742052,-0.0045658057,0.009925819,-0.061982065,0.06666853,-0.085638665,-0.008682048,0.016528588,-0.015443429,0.040419903,0.0059123226,-0.04848474,0.026133329,-0.042095724,-0.06860905,-0.033551272,-0.06492134,0.019667841,-0.04917464,-0.0096588,-0.10072659,-0.07769663,0.03221359,0.019174514,0.039727442,0.025392585,-0.016384942,0.0024048705,-0.09175566,-0.03225071,0.0066428655,0.10759633,-0.04011207,0.031578932,0.06299788,0.061487168,0.048043367,-0.0047893273,-0.054848563,-0.06647676,-0.027905045,-0.055799212,0.028914401,0.04013868,0.050728165,-0.0063177645,-0.018899892,0.008193828,0.025991635,-0.08009935,0.044058595,-0.046858713,0.072079815,0.046664152,0.019002488,-0.018447064,-0.15560018,-0.050175466,0.001016439,-0.0035773942,-0.025972001,0.047064543,0.01866733,0.0049167247,-0.052880444,-0.029235922,-0.024581103,0.040634423,0.095990844,-0.019483034,-0.02325509,0.056078408,0.09241045,-0.03079215,-0.023518562,-0.08394134,0.03326668,0.008070111,0.14776507,0.030338759,-0.01846056,0.009517991,0.0034727904,0.007246884,0.015436005,0.058226254,-0.037932027,-0.04309255,0.09766471,0.014914252,0.03149386,0.10146584,0.009303289,0.05649276,0.04743103,-0.016993523,0.054828145,0.033858124,-0.059207607,-0.027288152,0.09254907,0.07817234,-0.047911037,-0.023988279,-0.067968085,-0.03140125,-0.02434741,-0.017226815,0.050405838,0.048384074,0.10386314,-0.05366119,-0.048218876,0.022471255,-0.04470827,-0.055776954,0.0146418335,-0.03505756,0.041757654,0.0076765255,0.0637766}
 3 | {-0.06530473,0.043326367,0.027487691,-0.012605501,-0.003679171,0.0068843057,0.093755856,-0.018192727,-0.038994554,0.060702052,0.047350235,0.0015797003,-0.026038624,0.029946782,0.053223953,-0.009188536,-0.012273773,0.07512682,-0.1220027,0.024623549,0.040207546,-0.061494265,-0.0016338134,-0.096063755,-0.020626824,-0.0008177105,0.025736991,0.08205663,-0.064413406,-0.10329614,-0.050153203,0.022038238,-0.011629073,-0.03142779,0.09684598,-0.045188677,-0.032773193,0.041901052,0.032470446,0.06218501,0.00056252955,-0.03571358,0.030095506,-0.09239761,-0.020187493,-0.00932361,-0.08373726,-0.053929392,0.09724756,-0.032078817,0.02658544,0.009965162,0.07477913,0.05487153,0.023828406,0.06263976,0.06882497,0.08249143,0.062069558,0.08915651,-0.005154778,0.056259956,-0.13729677,0.08404741,0.07149277,0.04482675,-0.058625933,0.0034976404,-0.030747578,0.004520399,0.0007449915,9.660358e-05,-0.022526976,0.11449101,-0.043607008,0.026769284,0.021050733,0.05854427,-0.042627476,-0.022924222,-0.059794623,-0.037738875,-0.018500011,0.017315088,-0.00020744087,-0.0016206327,0.013337528,-0.022439854,-0.0042932644,-0.04706647,-0.06771751,-0.040391076,0.0638978,-0.031776994,0.011536817,-0.04593729,0.08626801,0.0016808647,-0.0046028513,0.13702579,0.02293593,0.043189116,-0.0073873955,-0.06097065,-0.019305069,-0.025651531,0.043129053,-0.033460874,0.03261353,-0.022361644,-0.07769732,-0.021210406,-0.020294553,-0.044899672,0.083500296,0.038056396,-0.052046232,-0.03215008,-0.028185,0.041909587,0.016012225,-0.0058236965,0.021344814,-0.037620485,0.07454872,-0.03517924,0.086520284,0.096695796,0.0937938,-0.04190071,0.072271764,-0.07022541,0.01583733,-0.0017275782,-0.05280332,-0.005904967,-0.046241984,-0.024421731,0.09988276,-0.0077029592,-0.04107849,-0.091607556,0.033811443,-0.1323201,-0.015927043,0.011014193,-0.039773338,0.033963792,-0.053305525,-0.005038948,-0.024107914,-0.0079898145,0.039604105,0.009226985,0.0010978039,-0.015565131,-0.0002796709,0.037623808,-0.059376597,0.015390821,-0.07600872,-0.008280972,0.023050148,0.0777234,0.061332665,-0.13979945,-0.009342198,0.012803744,0.049805813,-0.03578894,-0.05038778,0.048912454,0.032017626,0.015345221,0.10369494,-0.048897773,-0.054201737,-0.015793057,0.08130064,-0.064783126,0.074246705,-0.06964914,-0.025839275,0.030869238,0.06357789,-0.028754702,-0.02960897,-0.04956488,0.030501548,0.005857936,-0.023547728,0.03717584,0.0024309678,0.066338174,-0.009775384,-0.030799516,-0.028462514,-0.058787093,-0.051071096,-0.048674088,0.011397957,0.07817651,-0.03227047,0.027149512,-0.0030777291,0.061677814,0.0025318298,-0.027110869,-0.0691719,-0.033963803,-0.0648151,-0.033951994,-0.0478505,0.0016642202,-0.019602248,-0.030472266,0.015889537,-0.0009066139,0.032841947,0.021004336,-0.029254122,-0.09597239,-0.04359093,-0.15422617,-0.016366383,-0.059343938,-0.064871244,0.07659653,0.023196936,-0.021893008,0.080793895,-0.05248758,0.018764181,0.0008353451,-0.03318359,-0.04830206,-0.05518034,0.038481984,0.06544077,0.019498836,-0.054670736,0.040052623,-0.028875519,-0.047129385,-0.03614192,-0.012638911,-0.0042204396,0.013685266,-0.047130045,0.11024768,0.07135732,-0.017937008,-0.040911496,0.09008783,0.039298594,0.042975742,-0.08974752,0.08711358,-0.021977019,0.051495675,0.0140351625,-0.053809136,-0.08241595,0.04982693,-0.020355707,0.017629888,-0.039196398,0.08688628,-0.051167585,-0.029257154,0.009161573,-0.0021740724,0.027258197,0.015352816,-0.07426982,0.022452697,-0.041628033,-0.023250584,-0.051996145,-0.031867135,-0.01930267,-0.05257186,0.032619886,-0.08220233,-0.017010445,0.038414452,-0.02268424,0.007727591,0.0064041745,-0.024256933,0.0028989788,-0.06191567,-0.020444075,-0.010515549,0.08980986,-0.020033991,0.009208651,0.044014987,0.067944355,0.07915397,0.019362122,-0.010731527,-0.057449125,-0.007854527,-0.067998596,0.036500365,0.037355963,-0.0011789168,0.030410502,-0.012768641,-0.03281059,0.026916556,-0.052477527,0.042145997,-0.023683913,0.099338256,0.035008017,-0.029086927,-0.032222193,-0.14743629,-0.04350868,0.030494612,-0.013000542,0.021753347,0.023393912,0.021320568,0.0031570331,-0.06008047,-0.031103736,0.030275675,0.015258714,0.09004704,0.0033432578,-0.0045539658,0.06602429,0.072156474,-0.0613405,-0.047462273,-0.057639644,-0.008026253,0.03090332,0.12396069,0.04592149,-0.053269017,0.034282286,-0.0045666047,-0.026025562,0.004598449,0.04304216,-0.02252559,-0.040372007,0.08094969,-0.021883471,0.05903653,0.10130699,0.001840184,0.06142003,0.004450253,-0.023686321,0.014760433,0.07669066,-0.08392746,-0.028447477,0.08995419,0.028487092,-0.047503598,-0.026627144,-0.0475691,-0.069141485,-0.039571274,-0.054866526,0.04417342,0.08155949,0.065555565,-0.053984754,-0.04142323,-0.023902748,0.0066344747,-0.065118864,0.02183451,-0.06479133,0.010425607,-0.010283142,0.0940532}

You can see the near 2.5x speedup when generating 3 embeddings in a batch, because the model weights only need to be streamed from GPU RAM to the processors a single time. You should consider batch sizes from 10-100 embeddings at a time when do bulk operations to improve throughput and reduce costs.

Scalability

PostgresML serverless instances have access to multiple GPUs that be used simultaneously across different PostgreSQL connections. For large jobs, you may want to create multiple worker threads/processes that operate across your dataset in batches on their own Postgres Connection.