merge

Upsert / merge tables in SQLite

泪湿孤枕 提交于 2020-03-04 18:57:29
问题 I have created a database using sqlite3 in python that has thousands of tables. Each of these tables contains thousands of rows and ten columns. One of the columns is the date and time of an event: it is a string that is formatted as YYYY-mm-dd HH:MM:SS , which I have defined to be the primary key for each table. Every so often, I collect some new data (hundreds of rows) for each of these tables. Each new dataset is pulled from a server and loaded in directly as a pandas data frame or is

rxjs合并数据流操作符

限于喜欢 提交于 2020-03-04 11:01:30
一、concat首尾相连 工作方式: 当第一个Observable对象complete之后,concat就会去subscribe第二个Observable对象获取数据,把同样的数据传给下游。 直到最后一个Observable完结之后,concat产生的Observable也就完结了。 import { of,concat } from 'rxjs'; ... const source1$ = of(1,2,3) const source2$ = of(4,5,6) const source$ = concat(source1$,source2$) source$.subscribe( console.log, null, ()=>console.log('complete') ) ... 输出结果 二、merge:先到先得快速通过 工作方式: 第一个Observable对象不完结,并不影响下游的observable对象,随先得到,就先输出。 当所有的Observable对象完结了,merge才会完结自己的Observable对象。 import { timer, merge } from 'rxjs'; import 'rxjs/add/operator/map' ... // 从第0毫秒开始,每隔1000毫秒产生一个数据,依次是:0A、1A、2A.... const

海量数据处理

心已入冬 提交于 2020-03-04 00:37:21
海量数据处理,思路有hash、bitmap、merge、堆、Top K(BFPRT算法)、trie树、布隆过滤器。 hash:对很多数据进行hash,然后取余一个正整数n,可以分成n份不同的数据组,每份数据组中数据通过hash算法得到相同的下标,所以相同的数据一定会分在同一数据组中。 bitmap:为操作,如果为每个数据分配n个bit,那么每个数据可最多有2^n种情况,在很多数据的情况下,bitmap总长可设为很大并间隔n位分给一格数据。 merge:一个很大数据量数据排序,分成不同数据组,每组内进行排序,不同组之间进行merge排序。 堆排序:数据量大,而只需要知道最大或最小的几个,可以用堆排序。 Top K:https://blog.csdn.net/laojiu_/article/details/54986553 速度很快(类似快排选一个基准,不过这个基准选取方式为:n个数据5个一组,每个排序得到组内中位数,各组之间中位数排序得到组间中位数,该中位数为基准,大于该值一边,小于的另一边,则最终左边和右边都至少有3/10的数据量,递归下去最后得到想要的位置)。 trie数(字典树):用于去重字符串。字典树每个树枝(两个节点之间连线)一个字符,从根节点到叶子节点构成了不同的路径,每个路径唯一地表示了一个字符串。 布隆过滤器:使用了bitmap和hash,一个数据散列出n个特征点

LeetCode 21 链表 Merge Two Sorted Lists

放肆的年华 提交于 2020-03-04 00:21:20
LeetCode 21 链表 Merge Two Sorted Lists LeetCode Merge two sorted linked lists and return it as a new list. The new list should be made by splicing together the nodes of the first two lists. Example: Input: 1->2->4, 1->3->4 Output: 1->1->2->3->4->4 代码: 递归 class Solution { public ListNode mergeTwoLists(ListNode l1, ListNode l2) { if(l1 == null) return l2; if(l2 == null) return l1; if(l1.val < l2.val){ l1.next = mergeTwoLists(l1.next, l2); return l1; }else{ l2.next = mergeTwoLists(l1, l2.next); return l2; } } } 迭代 class Solution { public ListNode mergeTwoLists(ListNode l1, ListNode l2) { ListNode

Repetitive merges in GIT. How does it calculate differences?

点点圈 提交于 2020-03-03 03:05:49
问题 I've been doing a research of trying to understand how does the GIT merge works. I know there are several merge types as recursive, octopus, etc. I figured out that the resolve / recursive is used the most commonly. And that recursive merge is only useful when there are several common ancestors / bases. However, I couldn't find which algorithm is used (or how the ancestor should be calculated) with repetitive merges to the master from the branch. A simple example. Let's create an empty

Repetitive merges in GIT. How does it calculate differences?

我的未来我决定 提交于 2020-03-03 03:02:30
问题 I've been doing a research of trying to understand how does the GIT merge works. I know there are several merge types as recursive, octopus, etc. I figured out that the resolve / recursive is used the most commonly. And that recursive merge is only useful when there are several common ancestors / bases. However, I couldn't find which algorithm is used (or how the ancestor should be calculated) with repetitive merges to the master from the branch. A simple example. Let's create an empty

git基础教程(四)

拥有回忆 提交于 2020-03-03 01:23:50
4. 多人单分支协作操作 4.1 不同的人修改了不同的文件 在一台电脑上,通过git clone 命令获取到远端的代码 git clone [远端的http链接或者ssh链接] #克隆的时候指定目录 git clone [远端的http链接或者ssh链接] [文件夹名称] #但如果是通过git clone将仓库克隆下来的 会默认生成一个origin的简写指向远程仓库 假设现在第一台电脑 commit 以后,push到远端 第二台电脑进行修改commit以后,也向远端提交,则会发现报错 #这是因为远端有比本地更新的commit #所以需要先将远端 fetch 到本地 再次进行push发现还是会报错,如下图所示 利用 git merge 仓库名分支名 进行本地和远端的分支合并 最后再次尝试进行向远端推送 补充 #下面两条命令是成功的关键 git fetch 与 git merge 仓库名\分支名 #可以将以上两条命令用一条命令替代 git pull 4.2 不同人修改了同一文件的不同区域 与上一种情况类似 #下面两条命令是成功的关键 git fetch 与 git merge 仓库名/分支名 #可以将以上两条命令用一条命令替代 git pull 4.3 不同人修改了同一文件的同一区域 报错如下 这时候如果进行 git merge up to date 说明远端已经改变,需要进行

oracle的merge into 的用法

折月煮酒 提交于 2020-03-02 19:18:29
使用背景: ORACLE 数据操作的时候,需要通过一个表去 UPDATE/INSERT 更新另一个表的数据,就可以使用 MERGE INTO 语法了。 这是 ORACLE9i 新加入的语法。 语法: MERGE INTO 表名 A USING (表、视图、查询语句) B ON (条件) WHEN MATCHED THEN UPDATE SET A.col1 = B.col_val1, A.col2 = B.col_val2 WHEN NOT MATCHED THEN INSERT (A.column_list) VALUES (B.column_values); 示例: 1.USING(表) MERGE INTO a08 a USING a08_temp t ON (a.a0800 = t.a0800) WHEN MATCHED THEN UPDATE SET a.A0000=t.A0000,a.A0801A=t.A0801A where t.imprecordid=impid WHEN NOT MATCHED THEN INSERT (a.A0000,a.A0800) VALUES (t.A0000,t.A0800) where t.imprecordid=impid; 2.USING(查询语句) MERGE INTO table_name T1 USING (SELECT '1

Intersection of multiple pandas dataframes

坚强是说给别人听的谎言 提交于 2020-03-01 08:51:50
问题 I have a number of dataframes (100) in a list as: frameList = [df1,df2,..,df100] Each dataframe has the two columns DateTime , Temperature . I want to intersect all the dataframes on the common DateTime column and get all their Temperature columns combined/merged into one big dataframe: Temperature from df1, Temperature from df2, Temperature from df3, .., Temperature from df100. (pandas merge doesn't work as I'd have to compute multiple (99) pairwise intersections). 回答1: Use pd.concat , which

Intersection of multiple pandas dataframes

淺唱寂寞╮ 提交于 2020-03-01 08:48:27
问题 I have a number of dataframes (100) in a list as: frameList = [df1,df2,..,df100] Each dataframe has the two columns DateTime , Temperature . I want to intersect all the dataframes on the common DateTime column and get all their Temperature columns combined/merged into one big dataframe: Temperature from df1, Temperature from df2, Temperature from df3, .., Temperature from df100. (pandas merge doesn't work as I'd have to compute multiple (99) pairwise intersections). 回答1: Use pd.concat , which