AI智能
改变未来

pymongo insert_many 批量插入的实例

我就废话不多说了,大家还是直接看代码吧~

lt=client.fangjia.district_stat_all_0416dl = dt.find(query)bf=[]for m in dl:bf.append(m)if len(bf)==20000:lt.insert_many(bf)bf=[]lt.insert_many(bf)

补充知识:Python操作Mongodb插入数据的两种方法:insert_one()与insert_many()

代码说明:

将mysql中table的数据插入到mongodb里

sys.setdefaultencoding(\'utf8\')import webfrom pymongo import MongoClientclass getPltfList(object):def __init__(self):self.db1 = web.database(dbn=\'mysql\', db = \'episode\', user = \'root\', pw= \'abc111--\', host = \'127.0.0.1\')self.db1.printing = Falseself.db2 = web.database(dbn=\'mysql\', db = \'episode_soc\', user = \'root\', pw= \'abc111--\', host = \'127.0.0.1\')self.db2.printing = Falseself.db3 = web.database(dbn=\'mysql\', db = \'cl_episode\', user = \'root\', pw= \'abc111--\', host = \'127.0.0.1\')self.db3.printing = Falseself.db4 = web.database(dbn=\'mysql\', db = \'cl_episode_soc\', user = \'root\', pw= \'abc111--\', host = \'127.0.0.1\')self.db4.printing = Falsedef __call__(self):return self.createPltfList()def createPltfList(self):self.ckpltfList = list(self.db1.query(\'select name, ip from EPISODE_PLTF_INFO order by id DESC\'))self.ckpltfList += list(self.db2.query(\'select name, ip from EPISODE_PLTF_INFO order by id DESC\'))self.clpltfList = list(self.db3.query(\'select name, ip from EPISODE_PLTF_INFO order by id DESC\'))self.clpltfList += list(self.db4.query(\'select name, ip from EPISODE_PLTF_INFO order by id DESC\'))return self.ckpltfList,self.clpltfListif __name__ == \'__main__\' :mpList = list()flag = 0tmp = 0sum1 = 0sum2 = 0pltfList = getPltfList()()client = MongoClient(\"127.0.0.1\", 27017)mdb = client.episodecollection = mdb.pltf_basic_info# print (type(pltfList[1])) #listresult= list()for pltf_my in pltfList[0]:pltf_mog = collection.find()for record in pltf_mog:IP = record.get(\'Cfg\').get(\'Debug_IP\')Name = record.get(\'Cfg\').get(\'Register_Name\')# print IP, Nameif IP == pltf_my[\'ip\'] and Name == pltf_my[\'name\'] :flag = 1# print IP, Namebreakelse:flag = 0if flag == 0 :data1 = {\"Cfg\" : {\"Debug_IP\" : pltf_my[\'ip\'],\"Register_Name\":pltf_my[\'name\'], \"Site\":\"SH-CK\"} }# print data1result.append((data1))# collection.insert_one(data1)# collection.delete_one(data)sum1 = sum1+1# print len(result)# collection.insert_many(result)for pltf_my in pltfList[1]:pltf_mog = collection.find()for record in pltf_mog:IP = record.get(\'Cfg\').get(\'Debug_IP\')Name = record.get(\'Cfg\').get(\'Register_Name\')if pltf_my[\'ip\'].encode(\"utf-8\") == IP.encode(\"utf-8\") and pltf_my[\'name\'].encode(\"utf-8\") == Name.encode(\"utf-8\") :tmp = 1# print IP, Namebreakelse:tmp = 0if tmp == 0 :data2 = {\"Cfg\" : {\"Debug_IP\":pltf_my[\'ip\'],\"Register_Name\":pltf_my[\'name\'], \"Site\":\"SH-CL\"} }# print data2result.append((data2))# collection.insert_one(data2)# collection.delete_one(data)sum2 = sum2+1collection.insert_many(result)print sum1,sum2

刚开始的时候我使用的是insert_one()方法,一条一条的插入到mongodb的集合里,但是计算出的sum有出入。

在调试的过程中我发现:

注释掉 # collection.insert_one(data1) # collection.insert_one(data2)

计算出来的sum1 = 193 sum2 = 222 这是合理的,因为ck_mysql里有193条记录,cl_mysql里有234条记录,mongod里有总共有15条,但是12条是与cl_mysql重复,所以正确。

但是当我去掉注释使用 collection.insert_one(data1) collection.insert_one(data2)时,打印出的sum1=181 sum2 = 213

也就是说少了几个数据,我不知道去哪了。

想了好多办法,于是采用insert_many()的方法插入。先定义一个list(),将每个数据(数据的类型是dict)追加到list里:reslult.append(data1/data2),最后result里就会含有所有的数据,一起插入。

于是乎,问题解决了。

可是我还是很困惑insert_one()哪里出问题了!!!!!!

以上这篇pymongo insert_many 批量插入的实例就是小编分享给大家的全部内容了,希望能给大家一个参考

您可能感兴趣的文章:

  • 解决Pymongo insert时会自动添加_id的问题
  • python使用pymongo操作mongo的完整步骤
  • Python3中使用PyMongo的方法详解
赞(0) 打赏
未经允许不得转载:爱站程序员基地 » pymongo insert_many 批量插入的实例