upsert

I use Talend to load data into a sql-server database. It appears that the weakest point of my job is not the dataprocessing, but the effective load in my database, which is not faster than 17 rows/sec. The funny point is that I can launch 5 jobs in the same time, and they'll all load at 17rows/sec . What could explain this slowness and how could I improve the speed? Thanks New informations: The transfer speed between my desktop and the server is about 1MByte My job commits every 10 000 I use sql server 2008 R2 And the schema I use for my jobs is like this: Database INSERT OR UPDATE methods are

I have a record that I want to exist in the database if it is not there, and if it is there already (primary key exists) I want the fields to be updated to the current state. This is often called an upsert . The following incomplete code snippet demonstrates what will work, but it seems excessively clunky (especially if there were a lot more columns). What is the better/best way? Base = declarative_base() class Template(Base): __tablename__ = 'templates' id = Column(Integer, primary_key = True) name = Column(String(80), unique = True, index = True) template = Column(String(80), unique = True)

In this SO answer about Entity Framework and MERGE, the example for how to code it is this: public void SaveOrUpdate(MyEntity entity) { if (entity.Id == 0) { context.MyEntities.AddObject(entity); } else { context.MyEntities.Attach(entity); context.ObjectStateManager.ChangeObjectState(entity, EntityState.Modified); } } This assumes that you already know if the entity that you want to upsert exists or not; in this case you check entity.Id . But what if you don't know if the item exists or not? For instance, in my case, I'm importing records from a vendor into my database, and a given record may

When you are upserting a row (PostgreSQL >= 9.5), and you want the possible INSERT to be exactly the same as the possible UPDATE, you can write it like this: INSERT INTO tablename (id, username, password, level, email) VALUES (1, 'John', 'qwerty', 5, 'john@mail.com') ON CONFLICT (id) DO UPDATE SET id=EXCLUDED.id, username=EXCLUDED.username, password=EXCLUDED.password, level=EXCLUDED.level,email=EXCLUDED.email Is there a shorter way? To just say: use all the EXCLUDE values. In SQLite I used to do : INSERT OR REPLACE INTO tablename (id, user, password, level, email) VALUES (1, 'John', 'qwerty',

I have the following Entity Attribute value table : CREATE TABLE key_value_pair ( id serial NOT NULL PRIMARY KEY, key varchar(255) NOT NULL, value varchar(255), is_active boolean ); CREATE UNIQUE INDEX key_value_pair_key_if_is_active_true_unique ON key_value_pair (key) WHERE is_active = true; Sample entries in this table are : id | key | value | is_active ----+-------------+-------+----------- 1 | temperature | 2 | f 2 | temperature | 12 | f 3 | temperature | 15 | f 4 | temperature | 19 | f 5 | temperature | 23 | t (5 rows) Thus, at any point in time, for any given key, only 1 true is_active