postgresql

python爬虫爬取链家二手房信息

由 …衆ロ難τιáo~ 提交于 2021-02-09 10:00:58
#coding=utf-8 import requests from fake_useragent import UserAgent from bs4 import BeautifulSoup import json import csv import time # 构建请求头 userAgent = UserAgent() headers = { 'user-agent': userAgent .Chrome } # 声明一个列表存储字典 data_list = [] def start_spider(page): #设置重连次数 requests.adapters.DEFAULT_RETRIES = 15 s = requests.session() #设置连接活跃状态为False s.keep_alive = False #爬取的url,默认爬取的南京的链家房产信息 url = 'https://nj.lianjia.com/ershoufang/pg{}/'.format(page) # 请求url resp = requests.get(url, headers=headers,timeout=10) # 讲返回体转换成Beautiful soup = BeautifulSoup(resp.content, 'lxml') # 筛选全部的li标签

Python爬取链家二手房源信息

由 杀马特。学长 韩版系。学妹 提交于 2021-02-09 09:53:35
爬取链家网站二手房房源信息，第一次做，仅供参考，要用scrapy。 import scrapy,pypinyin,requests import bs4 from ..items import LianjiaItem class LianjiaSpider(scrapy.Spider): name = 'lianjia_dl' allowed_domains = ['www.lianjia.com'] start_urls = [] url_0 = 'https://www.lianjia.com/city/' res = requests.get(url_0) bs_cs = bs4.BeautifulSoup(res.text,'html.parser') xinxi_cs = bs_cs.find_all('div',class_='city_province') for data_cs in xinxi_cs: cs_s = data_cs.find('ul').find_all('li') for cs_1 in cs_s: yess = cs_1.find('a')['href'] if yess.find('fang')>=0: #若fang字符串在yess中，则yess.find('fang')是大于等于0的，显示在字符串中的位置 continue else:

psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility

由 …衆ロ難τιáo~ 提交于 2021-02-09 09:22:15
问题 I switched my postgres version from 9.2.24 to 9.6 because I need jsonb compatibility along with other recent features. I am running a vm on centos 7. I decided to wipe all existing instances of postgres (which are almost empty since it is staging) and then installed 9.6. However, once I enter the postgres shell I received the following error when I type psql : psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility I assume I need to change the path that psql is looking

psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility

由 此生再无相见时 提交于 2021-02-09 09:21:51
问题 I switched my postgres version from 9.2.24 to 9.6 because I need jsonb compatibility along with other recent features. I am running a vm on centos 7. I decided to wipe all existing instances of postgres (which are almost empty since it is staging) and then installed 9.6. However, once I enter the postgres shell I received the following error when I type psql : psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility I assume I need to change the path that psql is looking

cannot extract elements from a scalar

由 寵の児 提交于 2021-02-09 09:20:13
问题 I have 2 tables company and contacts. Contacts has addresses JSONB column. I tried a select statement with a join on contacts.linked_to_company and using jsonb_array_elements(company.addresses) but I get error 'cannot extract elements from a scalar' which I understand is because some entries do have [null] in column address. I have seen answers to use coalesce or a CASE statement. Coalesce I could get to not work and CASE example is in the select statement how do use it in a join? Here is the

psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility

由 六眼飞鱼酱① 提交于 2021-02-09 09:18:04
问题 I switched my postgres version from 9.2.24 to 9.6 because I need jsonb compatibility along with other recent features. I am running a vm on centos 7. I decided to wipe all existing instances of postgres (which are almost empty since it is staging) and then installed 9.6. However, once I enter the postgres shell I received the following error when I type psql : psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility I assume I need to change the path that psql is looking

psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility

由 橙三吉。 提交于 2021-02-09 09:17:36
问题 I switched my postgres version from 9.2.24 to 9.6 because I need jsonb compatibility along with other recent features. I am running a vm on centos 7. I decided to wipe all existing instances of postgres (which are almost empty since it is staging) and then installed 9.6. However, once I enter the postgres shell I received the following error when I type psql : psql: symbol lookup error: psql: undefined symbol: PQsetErrorContextVisibility I assume I need to change the path that psql is looking

cannot extract elements from a scalar

由 狂风中的少年 提交于 2021-02-09 09:14:10
问题 I have 2 tables company and contacts. Contacts has addresses JSONB column. I tried a select statement with a join on contacts.linked_to_company and using jsonb_array_elements(company.addresses) but I get error 'cannot extract elements from a scalar' which I understand is because some entries do have [null] in column address. I have seen answers to use coalesce or a CASE statement. Coalesce I could get to not work and CASE example is in the select statement how do use it in a join? Here is the

Postgresql: query on jsonb column - index doesn't make it quicker

由 ∥☆過路亽．° 提交于 2021-02-09 08:34:31
问题 There is a table in Postgresql 9.6 , query on jsonb column is slow compared to a relational table, and adding a GIN index on it doesn't make it quicker. Table: -- create table create table dummy_jsonb ( id serial8, data jsonb, primary key (id) ); -- create index CREATE INDEX dummy_jsonb_data_index ON dummy_jsonb USING gin (data); -- CREATE INDEX dummy_jsonb_data_index ON dummy_jsonb USING gin (data jsonb_path_ops); Generate data: -- generate data, CREATE OR REPLACE FUNCTION dummy_jsonb_gen

Postgresql: query on jsonb column - index doesn't make it quicker

由 左心房为你撑大大i 提交于 2021-02-09 08:27:55
问题 There is a table in Postgresql 9.6 , query on jsonb column is slow compared to a relational table, and adding a GIN index on it doesn't make it quicker. Table: -- create table create table dummy_jsonb ( id serial8, data jsonb, primary key (id) ); -- create index CREATE INDEX dummy_jsonb_data_index ON dummy_jsonb USING gin (data); -- CREATE INDEX dummy_jsonb_data_index ON dummy_jsonb USING gin (data jsonb_path_ops); Generate data: -- generate data, CREATE OR REPLACE FUNCTION dummy_jsonb_gen
订阅 postgresql