wget

Centos 7 修改YUM镜像源地址为阿里云镜像地址

别说谁变了你拦得住时间么 提交于 2020-01-21 16:06:11
安装 wget yum install -y wget 进入目录 cd /etc/yum.repos.d/ 创建目录 mkdir backup 默认源配备份 mv C* backup/ 下载阿里云yum源 wget -O /etc/yum.repos.d/CenOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo 清除旧缓存 yum clean all 创建新缓存 yum makecache 更新yum yum update -y 来源: https://www.cnblogs.com/jockming/p/12221886.html

linux环境中,常用传输文件方法

三世轮回 提交于 2020-01-21 05:30:00
wget: 目标及其打开端口 python -m SimpleHTTPServer 8080 远程传输 wget 192.168.31.200:8080/文件路径(相对路径) scp: scp 源 目标 例如: scp mouse@192.168.31.200:/文件路径(绝对路径) 目标文件路径 如果是文件夹,要加-r 来源: CSDN 作者: G_Super_Mouse 链接: https://blog.csdn.net/G_Super_Mouse/article/details/103749079

Linux Shell curl 和 wget 使用代理IP

删除回忆录丶 提交于 2020-01-19 20:56:25
Linux Shell 提供两个非常实用的命令来爬取网页,它们分别是 curl 和 wget curl 和 wget 使用代理 curl 支持 http、https、socks4、socks5 wget 支持 http、https 代理示例: #!/bin/bash # # curl 支持 http、https、socks4、socks5 # wget 支持 http、https # # 米扑代理示例: # http://proxy.mimvp.com/demo2.php # # 米扑代理购买: # http://proxy.mimvp.com # # mimvp.com # 2015-11-09 # http代理格式 http_proxy=http://IP:Port # https代理格式 https_proxy=http://IP:Port {'http': 'http://120.77.176.179:8888'} curl -m 30 --retry 3 -x http://120.77.176.179:8888 http://proxy.mimvp.com/exist.php # http_proxy wget -T 30 --tries 3 -e "http_proxy=http://120.77.176.179:8888" http://proxy.mimvp

centos7.7误删除python2.7后导致yum不可用

两盒软妹~` 提交于 2020-01-19 01:58:47
相关报错界面: 分析: 网上给出的方法都是修改/usr/bin/yum文件的头部,把/usr/bin/python修改为/usr/bin/python2.x就可以了,但我的情况是我把旧的python已经卸载了,于是尝试重新安装2.7版本的python python版本下载地址:https://www.python.org/ftp/python/ 下载2.7.5 wget https://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.xz 解压缩 tar -xvf Python-2.7.5.tar.xz 创建路径安装路径 mkdir /usr/local/python 切换到安装路径执行 ./configure --prefix=/usr/local/python/ make make install 创建软连接 ln -s /usr/local/python/bin/python /usr/bin/python 安装完成以后依旧报原有的错误,和上图一样;这有点烦银了,继续找办法。。。 发现有博客讲下载python以及yum的rpm包,行不行先试一试, 创建脚本下载相关的rpm包 mkdir /opt/packages/python vim pull.sh #!/bin/bash wget http://vault.centos

linux多线程下载工具 mwget

南笙酒味 提交于 2020-01-18 02:05:45
使用wget下载文件时十分缓慢 可使用mwget进行加速 其中m代表多线程的意思 1.安装mwget wget http://jaist.dl.sourceforge.net/project/kmphpfm/mwget/0.1/mwget_0.1.0.orig.tar.bz2 tar -xjvf mwget_0.1.0.orig.tar.bz2 cd mwget_0.1.0.orig 执行./configure ./configure 如果出现 error: C++ compiler cannot create executables 说明没有安装c++编译器 安装一个c++编译器就可以了 yum install gcc-c++ 如果执行./configure 出现 configure: error: Your intltool is too old. You need intltool 0.35.0 or later. 需要安装0.35.0以上的版本 yum install intltool 然后做最后的安装 make make install 安装完毕后 可以使用mwget下载 mwget https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz mwget功能选项 mwget --help #执行后可以直接看到帮助说明

WGET download happens faster when executed through command line and slowly when executed through Java code

依然范特西╮ 提交于 2020-01-15 11:09:13
问题 I am downloading a file using WGET through a java code, which takes around 10 mins to download 20 MB file. But on executing the wget download through command line, the same file gets downloaded in 7 seconds at 10MbPs speed. Does anyone know why this is? How can I improve my Java code? Below is the code I have used to download a file using WGET. It takes around 10 minutes to download a 20 MB file. But when I run the wget command through the command line, it happens in seconds!! import java.io

WGET download happens faster when executed through command line and slowly when executed through Java code

爱⌒轻易说出口 提交于 2020-01-15 11:08:51
问题 I am downloading a file using WGET through a java code, which takes around 10 mins to download 20 MB file. But on executing the wget download through command line, the same file gets downloaded in 7 seconds at 10MbPs speed. Does anyone know why this is? How can I improve my Java code? Below is the code I have used to download a file using WGET. It takes around 10 minutes to download a 20 MB file. But when I run the wget command through the command line, it happens in seconds!! import java.io

Extract IP address from HTML document

馋奶兔 提交于 2020-01-15 05:51:19
问题 How can I print the IP address (86.23.215.130) of the following line? The entire file (not shown) is the stdout from a wget (hence HTML). Sounds easy, but I didn't manage. ... <tr><td align=center colspan=3 bgcolor="D0D0D0"><font face="Arial, Monospace" size=+3>86.23.215.130</font></td></tr> ... Thanks 回答1: Why sed? I believe grep is much better: grep -iohP '(?<=\x3e)([0-9]+\.){3}[0-9]+(?=\x3c)' file where \x3e means > and \x3c means < (ascii hex code) Although sed can do this, but it's not

Mirroring a webpage recursively after JS execution

末鹿安然 提交于 2020-01-14 06:12:17
问题 I'm trying to mirror a webpage recursively, e.g. getting all pages on one webpage. All webpages are in subfolders of just one folder, therefore I could easily mirror all webpages using wget: wget --mirror --recursive --page-requisites --adjust-extension --no-parent --convert-links https://www.example.com/ However, the page is mirrored before some JS scripts are executed, and those JS scripts don't get mirrored. I need to mirror them too, somehow, because they change the webpage's DOM. Another

Mirroring a webpage recursively after JS execution

为君一笑 提交于 2020-01-14 06:11:21
问题 I'm trying to mirror a webpage recursively, e.g. getting all pages on one webpage. All webpages are in subfolders of just one folder, therefore I could easily mirror all webpages using wget: wget --mirror --recursive --page-requisites --adjust-extension --no-parent --convert-links https://www.example.com/ However, the page is mirrored before some JS scripts are executed, and those JS scripts don't get mirrored. I need to mirror them too, somehow, because they change the webpage's DOM. Another