Nemo evaluator notebooks (#270)

* Updating Evaluator Notebooks

* Signing off

Signed-off-by: Chris Alexiuk <calexiuk@nvidia.com>

---------

Signed-off-by: Chris Alexiuk <calexiuk@nvidia.com>
Co-authored-by: Chris Alexiuk <c.s.alexiuk@gmail.com>

Author: chrisalexiuk-nvidia

nemo/Evaluator/EmbeddingAndRAG/NeMo_Evaluator_Retriever_and_RAG_Evaluation.ipynb

@@ -0,0 +1,20 @@
+{"_id": "doc1", "title": "Eiffel Tower - A French Icon", "text": "The Eiffel Tower is an iconic landmark of Paris, France. It was designed by the engineer Gustave Eiffel and built for the 1889 Exposition Universelle (World's Fair) to celebrate the 100th anniversary of the French Revolution.", "metadata": {}}
+{"_id": "doc2", "title": "The Great Wall of China - Ancient Protection", "text": "The Great Wall of China is a series of fortifications made of stone, brick, tamped earth, wood, and other materials, generally built along an east-to-west line across the historical northern borders of China to protect the Chinese states and empires against raids and invasions from various nomadic groups.", "metadata": {}}
+{"_id": "doc3", "title": "Taj Mahal - A Symbol of Love", "text": "The Taj Mahal is an ivory-white marble mausoleum on the south bank of the Yamuna river in the Indian city of Agra. It was commissioned in 1632 by the Mughal emperor, Shah Jahan (reigned 1628\u20131658), to house the tomb of his favourite wife, Mumtaz Mahal.", "metadata": {}}
+{"_id": "doc4", "title": "Machu Picchu - Lost City of the Incas", "text": "Machu Picchu is a 15th-century Inca citadel situated on a mountain ridge above the Sacred Valley in Peru. It is the most famous icon of Inca civilization, known for its sophisticated dry-stone walls that fuse huge blocks without the use of mortar.", "metadata": {}}
+{"_id": "doc5", "title": "The Colosseum - Ancient Roman Architecture", "text": "The Colosseum, also known as the Flavian Amphitheatre, is an oval amphitheatre located in the centre of the city of Rome, Italy. Built of concrete and sand, it is the largest amphitheatre ever built and is considered one of the greatest works of Roman architecture and engineering.", "metadata": {}}
+{"_id": "doc6", "title": "Petra - Ancient City of Jordan", "text": "Petra is a historical and archaeological city in southern Jordan. The city is famous for its rock-cut architecture and water conduit system. Another name for Petra is the Rose City due to the color of the stone out of which it is carved.", "metadata": {}}
+{"_id": "doc7", "title": "Chichen Itza - Ancient Mayan City", "text": "The Chichen Itza is a large pre-Columbian city built by the Maya civilization. The ruins of Chichen Itza are located in the northern center of the Yucatan Peninsula, in the Mexican state of Yucatan.", "metadata": {}}
+{"_id": "doc8", "title": "Pyramids of Giza - Ancient Egyptian Wonders", "text": "The Pyramids of Giza are the largest Egyptian pyramids, located on the Giza plateau on the outskirts of Cairo, Egypt. The pyramids, built as tombs for Pharaohs, are the only surviving structures of the Seven Wonders of the Ancient World.", "metadata": {}}
+{"_id": "doc9", "title": "Sistine Chapel - Renaissance Masterpiece", "text": "The Sistine Chapel is a chapel in the Apostolic Palace, the official residence of the Pope, in Vatican City. It is famous for its architecture and its decoration that was painted by various most famous artists of the High Renaissance, including Michelangelo.", "metadata": {}}
+{"_id": "doc10", "title": "Grand Canyon - Natural Wonder of the World", "text": "The Grand Canyon is a steep-sided canyon carved by the Colorado River in Arizona, United States. The Grand Canyon is 277 miles (446 km) long, up to 18 miles (29 km) wide and over a mile (1.6 km) deep.", "metadata": {}}
+{"_id": "doc11", "title": "Christopher Lloyd", "text": "Christopher Allen Lloyd (born October 22, 1938)[1] is an American actor, voice actor, and comedian. He is best known for his roles as Emmett \"Doc\" Brown in the Back to the Future trilogy, Judge Doom in Who Framed Roger Rabbit (1988), Merlock the Magician in DuckTales the Movie: Treasure of the Lost Lamp (1990), Uncle Fester in The Addams Family (1991) and its sequel Addams Family Values (1993), and Grigori Rasputin in Anastasia (1997).", "metadata": {}}
+{"_id": "doc12", "title": "Tobacco advertising", "text": "After World War II, cigarette companies advertised frequently on television programs. To combat this move by the cigarette companies, the Federal Communications Commission required television stations to air anti-smoking advertisements at no cost to the organizations providing such advertisements. In 1970, Congress took their anti-smoking initiative one step further and passed the Public Health Cigarette Smoking Act, banning the advertising of cigarettes on television and radio starting on January 2, 1971. After the television ban, most cigarette advertising took place in magazines, newspapers, and on billboards. However, in 1999 all cigarette billboard advertisements were replaced with anti-smoking messages, with some of these anti-smoking messages playing parodies of cigarette companies advertising figures and slogans. Since 1984, cigarette companies have also been forced to place Surgeon's General warnings on all cigarette packs and advertisements because of the passing of the Federal Cigarette Labeling and Advertising Act.[8] Restrictions on cigarette companies became even tighter in 2010 with the Family Smoking Prevention and Tobacco Control Act. The act prohibits tobacco companies from sponsoring sports, music, and other cultural events and also prevents the display of their logos or products on T-shirts, hats, or other apparel.[9] The constitutionality of both this act and the Food and Drug Administration's new graphic cigarette warning labels are being questioned under cigarette companies' first amendment rights.[10]", "metadata": {}}
+{"_id": "doc13", "title": "Beti Bachao, Beti Padhao Yojana", "text": "In 26 August 2016, Olympics 2016 bronze medallist Sakshi Malik was made brand ambassador for BBBP.[7]", "metadata": {}}
+{"_id": "doc14", "title": "List of Prison Break episodes", "text": "The first five seasons of Prison Break have been released on DVD and Blu-ray in Regions 1, 2, and 4. Each DVD boxed set includes all of the broadcast episodes from that season, the associated special episode, commentary from cast and crew, and profiles of various parts of Prison Break, such as Fox River State Penitentiary or the tattoo.[4][5][6][7] Prison Break is also available online, including iTunes, Amazon Video, and Netflix.[8][9] After the premiere of the second season of Prison Break, Fox began online streaming of the prior week's episode, though it originally restricted viewing to the United States.[10]", "metadata": {}}
+{"_id": "doc15", "title": "Atomic bombings of Hiroshima and Nagasaki", "text": "By August 1945, the Allies' Manhattan Project had produced two types of atomic bomb, and the 509th Composite Group of the United States Army Air Forces (USAAF) was equipped with the specialized Silverplate version of the Boeing B-29 Superfortress that could deliver them from Tinian in the Mariana Islands. Orders for atomic bombs to be used on four Japanese cities were issued on July 25. On August 6, one of its B-29s dropped a Little Boy uranium gun-type bomb on Hiroshima. Three days later, on August 9, a Fat Man plutonium implosion-type bomb was dropped by another B-29 on Nagasaki. The bombs immediately devastated their targets. Over the next two to four months, the acute effects of the atomic bombings killed 90,000\u2013146,000 people in Hiroshima and 39,000\u201380,000 people in Nagasaki; roughly half of the deaths in each city occurred on the first day. Large numbers of people continued to die from the effects of burns, radiation sickness, and other injuries, compounded by illness and malnutrition, for many months afterward. In both cities, most of the dead were civilians, although Hiroshima had a sizable military garrison.", "metadata": {}}
+{"_id": "doc16", "title": "Two-party system", "text": "There is general agreement that the United States has a two-party system; historically, there have been few instances in which third party candidates won an election. In the First Party System, only Alexander Hamilton's Federalist Party and Thomas Jefferson's Democratic-Republican Party were significant political parties. Toward the end of the First Party System, the Republicans dominated a one-party system (primarily under the Presidency of James Monroe). Under the Second Party System, the Democratic-Republican Party split during the election of 1824 into Adams' Men and Jackson's Men. In 1828, the modern Democratic Party formed in support of Andrew Jackson. The National Republicans were formed in support of John Quincy Adams. After the National Republicans collapsed, the Whig Party and the Free Soil Party quickly formed and collapsed. In 1854, the modern Republican Party formed from a loose coalition of former Whigs, Free Soilers and other anti-slavery activists. Abraham Lincoln became the first Republican president in 1860.", "metadata": {}}
+{"_id": "doc17", "title": "Equilateral triangle", "text": "An equilateral triangle is the most symmetrical triangle, having 3 lines of reflection and rotational symmetry of order 3 about its center. Its symmetry group is the dihedral group of order 6 D3.", "metadata": {}}
+{"_id": "doc18", "title": "The O.C. (season 4)", "text": "The fourth and final season of The O.C., an American teen drama television series, aired in the United States from November 2, 2006 to February 22, 2007 and consisted of sixteen episodes. The O.C's final season aired Thursdays at 9:00\u00a0p.m. ET in the United States on Fox, a terrestrial television network.[1] Fox tested a new timeslot by airing the second episode on Wednesday at 9:00\u00a0p.m. ET against ABC's Lost and CBS's Criminal Minds.[2] With Lost about to enter a three-month hiatus, Fox hoped that changing the timeslot for The O.C. would attract back viewers it had lost since the end of the previous season; however, the move was unsuccessful and the show returned to its Thursday timeslot.[3]", "metadata": {}}
+{"_id": "doc19", "title": "Crossing the Bar", "text": "The extended metaphor of \"crossing of bar\" represents traveling serenely and securely from life through death. The Pilot is a metaphor for God, whom the speaker hopes to meet face to face. Tennyson explained, \"The Pilot has been on board all the while, but in the dark I have not seen him\u00e2\u20ac\u00a6[He is] that Divine and Unseen Who is always guiding us.\"[1]", "metadata": {}}
+{"_id": "doc20", "title": "Protectionism", "text": "Protectionism is the economic policy of restricting imports from other countries through methods such as tariffs on imported goods, import quotas, and a variety of other government regulations. Proponents claim that protectionist policies shield the producers, businesses, and workers of the import-competing sector in the country from foreign competitors. However, they also reduce trade and adversely affect consumers in general (by raising the cost of imported goods), and harm the producers and workers in export sectors, both in the country implementing protectionist policies, and in the countries protected against.", "metadata": {}}

nemo/Evaluator/EmbeddingAndRAG/synthetic/corpus.jsonl

@@ -0,0 +1,101 @@
+query-id	corpus-id	score
+d4791e95319ed5799fe2e42e52bc947608626cf54828725cc98e19a490e31bb8	doc1	1
+c6329f64c45dceca6c4bd99dcd9d12e41dfe1cd8f654616ab319e9e9e31b8e99	doc1	1
+4d7474ae36a0a7a8555dd42c863e51e2f2c25614391876a65695cbfcfa36637c	doc1	1
+ba9889ab0567d3a7f477d12370efaaced2f77282fb321198eb0f4eadb13f1b27	doc1	1
+9e4a4c17c17223eae9b10749486319eca51fe04364409454c147a5613e2c3e98	doc1	1
+9988a93f1d281f8a4f109a63f08f4678ec4625375250901e14761bd394e5381a	doc2	1
+239dec3fb00a7d424e21b55a3b55b5877951c7ef6450d2d4de8fdc5cf39424fd	doc2	1
+672ec0ce4c048890428b405cf5a5cf2fb62c37da11eb32a5a5666786c9848e2f	doc2	1
+bc62643829abe4184f204299034d98ae936a3711b8fbb93f607f7eb44e4962b8	doc2	1
+f91cd8f9315b982471907638dc60f846bd54fbdc34db9ae6750395aedae24352	doc2	1
+4544bce08ed71d23edebf19ef59a86ce3db295cb9546291e262495944aedf616	doc3	1
+611bb43ce3321561149d1f513cca46d8a7bdb8c11c32e3cbdbaa6ad373592ebf	doc3	1
+9d476a2f512e7b758fafe9504dfcf2bc6e6794ee85b6fca2c530a6a9c60f0832	doc3	1
+801e8214b018bd8e0aac7e8631d7a82face81d483fbe3d6009d9278a946d7f68	doc3	1
+ea27f5a45db1ba616513dc69654d1975fcd60831eb87629b068c2333b2d4170a	doc3	1
+719cb39d8508cecd912286496cacf63c4e40b92fb19514738b591dcfc00de93f	doc4	1
+c301a19a6e44e7dbcf4199b795e268c12cb03980ba7dd0c1b0987e4124273fc8	doc4	1
+861cbf8c481828629aece860b032d73b15fa24ec32c7ca56f50463ed361fc226	doc4	1
+a8beced7938da062f56f99cb5aa0f591653f5604a45460978d79a3c7c4c4d1af	doc4	1
+8439305d8e992d0382bfd844ec1112f8e6fa74a06c6675a3a5df8ec283768558	doc4	1
+463d8b1341734939d23575b487076168f9d3189a3c8caf71d2102b3b260c2f35	doc5	1
+98968d1bf294786eb1b16249ae84e9c5bd61f8b2d198ffaece756c040c6fb920	doc5	1
+cef3453c377c4212cfe324f88c03d10a0f9adf39ef5414c9010e3fee616d2e16	doc5	1
+b054567594dc05b28e7a9285874a42b34d4975b3faa11831576223fa7acdeda8	doc5	1
+8fa174b629b8c6e50f3a76d8fdfcc7902f575ffa12624be813d5bfbd9f767831	doc5	1
+100507878129d6b20eaa8150a72bce36d4a60d08d5fc1aedf9d59054bce20da6	doc6	1
+903e62c8d08b3f58252ccda641d90ccb80bea056c56489bf8f0bb8e800fd13d0	doc6	1
+abd53764de3136706f539beb77d0d39c69e6e1617affd6822d1d61e5cb7ccc69	doc6	1
+482d31f3d23498a496719d5d461f86407a93ed344d922d045afb327dee0e2800	doc6	1
+bc29052d954a0821ab6221dae7cdadb471d2f02f5672aca529c5e077e4620857	doc6	1
+1f2cac557fd69325838a58f0402d7668092c444a2c5afef49e169db144668739	doc7	1
+32fe21d4640076008527e1318c3760e30317ba2d6f5196a00a738c7ddaaf637e	doc7	1
+4d951731a1df580f54afbb4ec2f9095c0e4133d490b061630e8c82a6baa457f2	doc7	1
+63697d4c54f7c5c6e2aa9eefbf561354f28cf842287550ce7e6b80a23b44c508	doc7	1
+383641b56f501cf1b4bca7512dc78e9df4b6208c776bb64cd910acaaa05a03fb	doc7	1
+4ab9b1148552aa354a95712e3bb78fa7625151e9727c22a9716eee55c8cd1ca5	doc8	1
+8a0f32e2bdda9140025f64fb77d82b8ed2c67b971cb4c44432ca1847a3e33ebc	doc8	1
+f3bf2d4d988d35bcd34440f4df27d81e705aa1e4250a44e1573f2426e4c672c9	doc8	1
+5f87ea96710eeadd6b9fce4b8fa0a31c4e4ad0c4deefa1d4f03c003b812ca35c	doc8	1
+4006d3300b1e5e882bde97c7a38cd57f11e3bea840c5d02c5106b2877eada5af	doc8	1
+0492bed59f5f2f477031319746855c36d341367dd0981cd1def258d4f6f233b2	doc9	1
+e24a3dc0487600b38b9224ff5eab454a3434cab46be17557967c63bf1d71bc41	doc9	1
+1bfc6624fca71593956762639a30f954c022e45161a59d7aa2c387451fb2628d	doc9	1
+1ca5ffac6744e0320a1d7d7d5ab6c106321a2a5ca950072faa330ea8849c5323	doc9	1
+5c4bc852ee60333f3dbe85c4a973da86767e84f3a696aba84ce093ef7ae1db07	doc9	1
+f8744a94668898ea7a9af97d04c870d73f9718a569f5c4ba0a0b46387c132924	doc10	1
+668a0071474dae0a9a07cc97322c8c43666c3d1dbda400527842d5313f85f81d	doc10	1
+212976ea90e81e91a3eb1032dbad007ae21ab3e36d3a197f470191651d8554a1	doc10	1
+ab31a5f5da3eb9b30e4597c8f2eeded3f4403c1372b6f8a74f7fd14f7b4ecd61	doc10	1
+e5dbb3daaf04133c91aee4ae5c23ad3eb1757a09bb476813ba1b8d744ec86853	doc10	1
+d6006420dc385e72a2e8eda25542df449150578a282fda6663269b021b66098c	doc11	1
+5a73841ef94dfaf35f5ac1b3fb6ba819cff3adeba1f45c9e1e3bdb683858577c	doc11	1
+8b861544082204ba43b1a68d3e1fc9d59ad248b728574706d74afbd40192f040	doc11	1
+c4b0c28cc818b06c14fd1135df72379ce5cf2631a490fcfa219935191ba08b84	doc11	1
+42a3f8e0d64feee3603b181022a106a97d71344720b923bc1f90af78964dd493	doc11	1
+e2836ccb6afb375121364f988119e252fae23780ee137f145de6da21ab53dafb	doc12	1
+d1e3c12606ef7743f57751e47e8e404ff2763f290121e393758d71f4a3aa9ca7	doc12	1
+1e5c00b6dc0b3efd1bcf172c4e310142ea3bdc44adf9855c7e6016c5e290fbdf	doc12	1
+b753b1f7764f19fb207c491283bfeb26c47d3c55ec946428cf76772ae36506d2	doc12	1
+af1aae944e5f4ccf2a95101c2cdf2ac638a72bda7fe88872cd5955c56996c48e	doc12	1
+c15597cc9a8f95e6e1353309e69590cc2d9ebcab1a999ce2b4e7e0a2493d38bf	doc13	1
+9ba6e043d95e03e67bfbf5099c7de9e6c4089679d3a72ae025c895e84b7ce45d	doc13	1
+5e546550dbec16a3f4cbe2ac034e5a71cbe80dad360091456d0b344ecc2d3153	doc13	1
+4dcbc0c9666769faf634cb3c5a8539f09d09774a702f70662b72ee84260c6a9a	doc13	1
+9e97131fc58c473f7ac7d3e0ed0f25212e7e5d773d02641159eab064b46a209f	doc13	1
+3512d9923d2f81059aa64d569c272fe33ced6f931c8c01fad46fd098a54c0e09	doc14	1
+6d7af57205637b8fa8cb7665619f5d36af09f8f56b06515aceee0813187404f7	doc14	1
+7e111e169cfef0f5c99cbabb522d72df2ed55134641aed68bbe9198a017a3f67	doc14	1
+28c42b4f42ca39358042db96a01aec297fa886e2d9a36cc84870fda089a6870b	doc14	1
+77ec725f54837f1d86a1dd2c52f433c2b83290aa4339593b7c775c131e6c5b06	doc14	1
+a95b549912c99a328763a45f0511b4344fc6549e8e90caf6bd9f5210e5f0661f	doc15	1
+f2b36e091bb21ef5903d17a80534859061406e40af0de87db33e13ec2387b832	doc15	1
+bfd509121ad8a90e0533026c08d452e1bd56f83db167e8157f221f29c152ee10	doc15	1
+500924f95b3ae6011ae7cf060eb40e93b9d2d62944760e7dbd407d6231c67523	doc15	1
+a4b087bdf981463fb0c9339414fbcb1af5a0962cb300a20bd58c945cc3e34dec	doc15	1
+62c6de75339d64b3c5b4d5f434f6b4e9d8b2625d88a39508d9aedfc3a1b39147	doc16	1
+a13a16a9168d8f18c6c8e636649797aa6067d1c870f6f7b57e9ea01ef7b7ec85	doc16	1
+57d13fd5c2ced7da0b4a93b52e0dc238564dacf567b8016238f484dc628e3fbb	doc16	1
+4fb124be5d39851dc01336dde383ed082ca2a2e0e333cd92e857921c7022f2b3	doc16	1
+e79e0a2a9de22158daf2c5037fcbc4ab3ca71bcddcb4818980e3d7ae9989bfa4	doc16	1
+9ce5f6e8874f507ee6c4df3a7b56ad620e3403b74b0e5bada827f5a79a2905b9	doc17	1
+555c5b1a2e9f6549d0fcdfb90b044d3760ac7f52e29ca6a23d071549b24cbaea	doc17	1
+b7e4e3a74de3a4ebd3cc5274dad55b64c304425212d9069d3ccaa24d8f30fdb0	doc17	1
+9499207380debdffa8c33f2ce6fa1a8189ff578707f62dfd916dc7d1b283eaa1	doc17	1
+90d4377aa058d6ef0124fbc61edde6ea9edb0efb7af5758322f9720e45434f12	doc17	1
+bb01d6a1a44ecc88fc81b47963eb585ce41d8cf3d9130b7af87c0fa16afc58fc	doc18	1
+d10bec3ed0b1de2d9e2d34d5f1b3f9a064f0c55ad94f614d2ea017386b9a170d	doc18	1
+1b16d1d166643ba00ec3e7a52ae5dfd17d7fd266817e0559f1f8e7ec1d91eefc	doc18	1
+12ddbc559c471313e736b6a136d2b40239200b97169979613aa5e13bd93e49a3	doc18	1
+227372a9d23586b741e7c969719d3e46a3845438ba9c71bab6777b1623c95c6d	doc18	1
+c78d6e1e3b459f5e484a61bfb174f482f5106aedabd41eab2321521ff98f5901	doc19	1
+e8eee05c41c2b11ca21e911a6efe892e4fb2423a888d31292a3db7d0ac4e0b5f	doc19	1
+5ddd182273afaad1041cce9ac0acd222b7e3f4e5f6d32a53dbc8dcb10006174e	doc19	1
+f77123640adc18f3c20d4545ca4efd137ff488d3c00e23edaeaf6047477d0454	doc19	1
+4934a02194e60bef81ae48e268589cb25a5a1d77a945fa8ab5db484f7901ed68	doc19	1
+a6cf02c4aac83a7ace43a93d3d65269666ec4d78a1e47f043b711f289e2b5a0a	doc20	1
+05114cb50594dadce7fffb81f84b82736cd1b9a945fbf1d0206ba4d6f9383b5f	doc20	1
+0ffed93e17e7077c9e7c31ed78cfbd9b49b3a015e81daccb239bc9f76f95131f	doc20	1
+74d7b7cbea4794a880cbfbf9f0d212b7c3e3f9f00657bac9ce6f363ac6947560	doc20	1
+e9973a6255565aee321cdc29ceea47a84aa5c392a49c017cad5c704f44d1ecc0	doc20	1