Entering an input, clicking it and retrieving particular information with Selenium [on hold]



  • 问题

    I\'m new to web scraping with Python. My intent is to retrieve the verb for a word of interest. For e.g. dictionary.com has definitions for different parts of speech for word, I would like to enter a word of interest and then hit the search icon, in the resulting page I would like to extract the information under the header \'verb\'.

    回答1:

    1

    To extract the information under verb header Induce WebDriverWait and presence_of_all_elements_located()

    Here is the code.

    from selenium import webdriver
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC
    
    driver=webdriver.Chrome()
    driver.get("https://www.dictionary.com/")
    WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH,"//button[contains(.,'Accept Cookies')]"))).click()
    elementsearch=WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.CSS_SELECTOR,"input[title='Search']")))
    elementsearch.send_keys("interest")
    elementsearch.submit()
    results=WebDriverWait(driver,20).until(EC.presence_of_all_elements_located((By.XPATH,"//span[@class='luna-pos'][contains(.,'verb')]/following::div[1]//div[@class='default-content']//div")))
    for item in results:
        print(item.text)
    
    results1=WebDriverWait(driver,20).until(EC.presence_of_all_elements_located((By.XPATH,"//span[@class='luna-pos'][contains(.,'verb')]/following::div[1]//div[@class='expandable-content']//div")))
    for item in results1:
        print(item.get_attribute("textContent"))
    
    

    Output on console:

    to engage or excite the attention or curiosity of:
    Mystery stories interested him greatly.
    to concern (a person, nation, etc.) in something; involve:
    The fight for peace interests all nations.
    to cause to take a personal concern or share; induce to participate: to interest a person in an enterprise.
    to cause to be concerned; affect.
    
    

    share|improve this answer

    answered 20 hours ago

    KunduKKunduK 11.4k22 gold badges33 silver badges2424 bronze badges

    • Why should the `waiting' happen? – Schlator 20 hours ago

    • 1

      Browser is taking time to load properly Not the code.The explicit wait is waiting until selenium find the required webelement. – KunduK 20 hours ago

    add a comment |



最新内容

  • 付费的商用数据库:

    Oracle,典型的高富帅; SQL Server,微软自家产品,Windows定制专款; DB2,IBM的产品,听起来挺高端; Sybase,曾经跟微软是好基友,后来关系破裂,现在家境惨淡

    免费的开源数据库:

    MySQL,大家都在用,一般错不了; PostgreSQL,学术气息有点重,其实挺不错,但知名度没有MySQL高; sqlite,嵌入式数据库,适合桌面和移动应用。

    Linux下,MySQL的配置文件默认存放在/etc/my.cnf或者/etc/mysql/my.cnf:

    mysql 命令

    1.启动:/etc/init.d/mysqld start

    2.停止:/etc/init.d/mysqld stop

    3. 重启: /etc/init.d/mysql restart

    来源:CSDN

    作者:幽雨雨幽

    链接:https://blog.csdn.net/ai2000ai/article/details/103080367

    read more
  • 第一点:scanf默认回车空格是输入不同组之间的间隔和结束符号。 也就是说他不会读取 空格 和 换行符。而是把他们当作一个 数据被读取完成的标志!

    例子

    两种输入方式

    第二点 若要scanf 读取到 换行符或者是空格 需要用 %c 来表示例如这样

    可以看到第一个换行符 和之后的空格全部读取到了。

    来源:https://www.cnblogs.com/LuoRuidiLoveMiku/p/11370308.html

    read more
  • I want to put an image from camera to database using this database code. What should be the code for the save button and the preview button?

    public class databases extends SQLiteOpenHelper { public static final String data = "student.db"; public static final String TABLE_NAME = "Student_TABLE"; public static final String COL_1 = "Fname"; public static final String COL_2 = "password"; public static final String COL_3 = "Fathers_name"; public static final String COL_4 = "Roll"; public static final String COL_5 = "Image"; public databases(Context context) { super(context, data, null, 1); } @Override public void onCreate(SQLiteDatabase db) { db.execSQL("create table " + TABLE_NAME + " (Fname TEXT,password TEXT,Fathers_name TEXT,Roll TEXT PRIMARY KEY,Image BLOB)"); }

    Now This is the activity code containing the respective buttons.

    public class home_page extends AppCompatActivity { Button lout,homepage,showperson,picture;static final int REQUEST_IMAGE_CAPTURE = 1; TextView showit; String userid,pwrd,r2,r3,r4; databases dbss; public void getterclass(String m) { userid=m; } @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_home_page); lout = (Button) findViewById(R.id.out); homepage = (Button) findViewById(R.id.home); showit=(TextView) findViewById(R.id.showtextdata); showperson=(Button)findViewById(R.id.showdata); picture=(Button)findViewById(R.id.images); dbss=new databases(home_page.this); getSupportActionBar().hide(); homepage.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Intent redirect=new Intent(home_page.this,home_page.class); startActivity(redirect); } }); lout.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Intent logout=new Intent(home_page.this,log_in.class); startActivity(logout); } }); showperson.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Cursor sh=dbss.givedata(userid); StringBuffer buf=new StringBuffer(); buf.append("First name :" + sh.getString(0) + "\n"); buf.append("password :" + sh.getString(1) + "\n"); buf.append("Father's name :" + sh.getString(2) + "\n"); buf.append("Roll :" + sh.getString(3) + "\n\n"); showit.setText(buf.toString()); } }); picture.setOnClickListener(new View.OnClickListener() { @Override public void onClick(View v) { Intent takePictureIntent = new Intent(MediaStore.ACTION_IMAGE_CAPTURE); if (takePictureIntent.resolveActivity(getPackageManager()) != null) { startActivityForResult(takePictureIntent, REQUEST_IMAGE_CAPTURE); } databases help=new databases(home_page.this); } });

    The getterclass() is nothing but to retrieve the personal information of the user into other activity however that isn't working as well when I try to display the data to in the same activity.

    You extract the image as a byte[] and then either convert that to the bytes, as a string of hexadecimal characters and then enclose that string in X' your_hexadecimal_string '

    e.g. it could be X'FFFEFDFCFBFA9........'

    and use that string as the value to insert via native SQL

    e.g. INSERT INTO your_table (image_column) VALUES(X'FFFEFDFCFBFA9........')

    Or you could use the SQliteDatbase insert convenience method, which takes a Contentvalues object as the 3rd parameter. You'd setup the Contentvalues object something along the lines of

    ContentValues cv = new Contentvalues(); cv.put("Image",your_image_as_a_byte_array); db.insert("the_table_name",null,cv);

    However, storing images can be troublesome and it is not recommended to store images, especially if they are over about 100k in size on average.

    If an image is near to or greater than 2M you will be unable to retrieve that image using the standard Android SQLite API as a CursorWindow has a limitation of 2M. Even at 1M you could perhaps get 1 image within a CursorWindow.

    The recommended method is to store the path or part of the path to the image and to retrieve the image via it's path when required.

    Example

    Saying that (the above), the following is an App that will either store the image in the Database if it is below 100K (public static final int MAX_FILE_SIZE = 100 * 1024;) or store the path and either retrieve the image via the path or the database according to what was stored.

    The App has two ListViews with the images descriptions (the rightmost includes the path to the image). Clicking on an item in either displays the image.

    e.g.

    enter image description here

    There are a number of images most over 100k, one funny.png is below 100k :-

    enter image description here

    The images have been placed into the Raw folder of the Res folder for this example, they are then copied to data/data/files/images folder (where represents the name of the App's package):-

    enter image description here

    The DatabaseHelper, DBHelper.java is :-

    public class DBHelper extends SQLiteOpenHelper { public static final String DBNAME = "images.db"; public static final int DBVERSION = 1; // The maximum size of an image that should be stored 100K public static final int MAX_FILE_SIZE = 100 * 1024; public static final String TB_IMAGE = "image"; public static final String COL_IMAGE_ID = BaseColumns._ID; public static final String COL_IMAGE_PATH = "image_path"; public static final String COL_IMAGE_NAME = "image_name"; public static final String COl_IMAGE_DESCRIPTION = "image_description"; public static final String COL_IMAGE_SIZE = "image_size"; public static final String COL_IMAGE_IMAGE = "image"; SQLiteDatabase mDB; /** * Construct DBHelper, note that it will open the database and * thus create it if it doesn't exist * @param context a context from the invoking activity */ public DBHelper(Context context) { super(context, DBNAME, null, DBVERSION); mDB = this.getWritableDatabase(); } /** * Create the table(s) * @param db */ @Override public void onCreate(SQLiteDatabase db) { String crtsql = "CREATE TABLE IF NOT EXISTS " + TB_IMAGE + "(" + COL_IMAGE_ID + " INTEGER PRIMARY KEY, " + COL_IMAGE_PATH + " TEXT UNIQUE, " + COL_IMAGE_NAME + " TEXT, " + COl_IMAGE_DESCRIPTION + " TEXT, " + COL_IMAGE_SIZE + " INTEGER, " + COL_IMAGE_IMAGE + " BLOB DEFAULT x'00'" + ")"; db.execSQL(crtsql); } @Override public void onUpgrade(SQLiteDatabase sqLiteDatabase, int i, int i1) { } /** * Return a Cursor with all the rows from the image table * @return The Cursor */ public Cursor getImageList() { return mDB.query(TB_IMAGE,null,null,null,null,null,null); } /** * Store an image row in the image table, noting that is the image * size is small than the max size that the image will be stored as a blob * otherwise a blob of 1 byte is stored due to the default value. * @param path the path to the image * @param description a description for the image * @return the id (rowid) of the row */ public long addImageFromPath(String path, String description) { ContentValues cv = new ContentValues(); File f = new File(path); InputStream is; // If the file doesn't exist don't store a row if (!f.exists()) { return -1; } // Always store the name, description, path and size cv.put(COL_IMAGE_NAME,f.getName()); cv.put(COl_IMAGE_DESCRIPTION,description); cv.put(COL_IMAGE_SIZE,f.length()); cv.put(COL_IMAGE_PATH,f.getAbsolutePath()); // If the size is less than the max then get the filestream // and convert to a byte[]. // Note if larger then the max file size the default x'00' blob // will be applied if (f.length() < MAX_FILE_SIZE) { byte[] buffer = new byte[(int) f.length()]; try { is = new FileInputStream(f); is.read(buffer); } catch (IOException e) { e.printStackTrace(); return -1; } cv.put(COL_IMAGE_IMAGE,buffer); } // Do the insert return mDB.insert(TB_IMAGE,null,cv); } /** * get the image as a bitmap from the DB if stored, otherwise get it from * the file, according to the id. * @param id the id of the row in the image table * @return the bitmap to be returned (note may be empty bitmap) */ public Bitmap getImage(long id) { byte[] ba = new byte[0]; // If the image is stored in the DB then extract and return the bitmap if (isStoredAsImage(id)) { return getImageAsBitMap(id); } // If not then get the respective row from the DB Cursor csr = mDB.query( TB_IMAGE, null, COL_IMAGE_ID+"=?", new String[]{String.valueOf(id)}, null, null, null ); // Prepare to convert the path to a file String path = ""; //<<<< default to empty path File f = new File(path); //<<< default to empty file // If a valid row was found get the path and File from the row if (csr.moveToFirst()) { path = csr.getString(csr.getColumnIndex(COL_IMAGE_PATH)); f = new File(path); } // done with the cursor so close it csr.close(); // If the file exists then return the Bitmap if (f.exists()) { return BitmapFactory.decodeFile(f.getAbsolutePath()); } // return an empty bitmap return BitmapFactory.decodeByteArray(ba,0,ba.length); } /** * Check to see if an image is stored in the DB, * note assumes anything less than 8 bytes isn't an image * @param id the id of the row in the image table * @return true if like an image is stored, otherwise false */ private boolean isStoredAsImage(long id) { boolean rv = true; byte[] ba = new byte[0]; // Get the respective row from the image table Cursor csr = mDB.query( TB_IMAGE, null, COL_IMAGE_ID+"=?", new String[]{String.valueOf(id)}, null, null, null ); // If a row was found get the blob into byte array ba // if not then ready to return false if (csr.moveToFirst()) { ba = csr.getBlob(csr.getColumnIndex(COL_IMAGE_IMAGE)); } else { rv = false; } // If the byte array ba is less then 8 bytes then ready to return false if (ba == null || ba.length < 8) { rv = false; } // done with the Cursor so close it csr.close(); // return the result return rv; } /** * get the image (assumes isStoredAsImage is used prior to invocation) * @param id the id of the respective row * @return the bitmap (may be 0 length) */ private Bitmap getImageAsBitMap(long id) { byte[] ba = new byte[0]; Bitmap bmp; Cursor csr =mDB.query( TB_IMAGE, null, COL_IMAGE_ID+"=?", new String[]{String.valueOf(id)}, null, null, null ); if (csr.moveToFirst()) { ba = csr.getBlob(csr.getColumnIndex(COL_IMAGE_IMAGE)); } csr.close(); return BitmapFactory.decodeByteArray(ba,0,ba.length); } }

    The Invoking activity MainActivity.java is :-

    public class MainActivity extends AppCompatActivity { public static final String IMAGES_DIRECTORY = "images"; private static File images_file; ArrayAdapter<String> mAdapter; ListView mListView01, mListView02; ArrayList<String> mImages; CursorAdapter mCsrAdapter; Cursor mCsr; ImageView mImageView; DBHelper mDBHlpr; @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); setContentView(R.layout.activity_main); // get the View/Viewgroup IDs mListView01 = this.findViewById(R.id.listview001); // File List mListView02 = this.findViewById(R.id.listview002); // DB List mImageView = this.findViewById(R.id.imageview001); // Image display // get an instance of the DBHelper mDBHlpr = new DBHelper(this); // Copy images from raw folder to data/data/<package>/Files/images // Also store all the images in the Database (or not depedning upon size) getImagesFile(this); if (getImagesCount() < 1) { loadRawImages(); storeImagesToDB(); } // Setup the two ListViews to display image name lists displayList(); displayListFromDB(); // setup the file list so that when an item is clicked the image is displayed mListView01.setOnItemClickListener(new AdapterView.OnItemClickListener() { @Override public void onItemClick(AdapterView<?> adapterView, View view, int i, long l) { String imagename = mListView01.getItemAtPosition(i).toString(); displayImage(imagename); } }); // setup the DB list so that when an item is clicked the image is displayed mListView02.setOnItemClickListener(new AdapterView.OnItemClickListener() { @Override public void onItemClick(AdapterView<?> adapterView, View view, int i, long l) { displayDBImage(l); } }); } /** * Store the images in the images folder to the DB giving then a calculated description * e.g. image1, image2 ..... */ private void storeImagesToDB() { File f = getImagesFile(this); File[] images = f.listFiles(); int imagecounter = 1; for (File img: images) { mDBHlpr.addImageFromPath(img.getPath(),"image" + String.valueOf(imagecounter++)); } } /** * return the directory/folder where the images are stored as a File * @param context a valid context * @return the number of images */ public static File getImagesFile(Context context) { if (images_file == null) { images_file = new File(context.getFilesDir().getPath() + File.separator + IMAGES_DIRECTORY); if (!images_file.exists()) { images_file.mkdirs(); } } return images_file; } /** * get the number of images * @return the number of images */ public static long getImagesCount() { File[] files = images_file.listFiles(new FileFilter() { @Override public boolean accept(File file) { return file.isFile(); } }); return (long) files.length; } /** * Setup/refresh the list of images according to the images folder * (left ListView) */ private void displayList() { if (mImages == null) { mImages = new ArrayList<>(); } else { mImages.clear(); } mImages.addAll(Arrays.asList(images_file.list())); if (mAdapter == null) { mAdapter = new ArrayAdapter<String>(this,android.R.layout.simple_list_item_1, mImages); mListView01.setAdapter(mAdapter); } else { mAdapter.notifyDataSetChanged(); } } /** * Set the image view according to the file * @param imageName the name of the image (as per the ListView) */ private void displayImage(String imageName) { File img = new File(images_file.getPath() + File.separator + imageName); if (img.exists()) { Bitmap bmp = BitmapFactory.decodeFile(img.getAbsolutePath()); mImageView.setImageBitmap(bmp); } } /** * Set the image view according to the image stored/referred to by the DB * @param id the id of the respective row in the image table */ private void displayDBImage(long id) { mImageView.setImageBitmap(mDBHlpr.getImage(id)); } /** * Setup/refresh the list of images as obtained from the DB (right listview) */ private void displayListFromDB() { mCsr = mDBHlpr.getImageList(); if (mCsrAdapter == null) { mCsrAdapter = new SimpleCursorAdapter( this, android.R.layout.simple_list_item_2, mCsr, new String[]{DBHelper.COL_IMAGE_NAME,DBHelper.COL_IMAGE_PATH}, new int[]{android.R.id.text1,android.R.id.text2}, 0 ); mListView02.setAdapter(mCsrAdapter); } else { mCsrAdapter.swapCursor(mCsr); } } /** * Load (copy from raw folder to images folder) all images */ private void loadRawImages() { Field[] fields = R.raw.class.getFields(); int resourceID = 0; String resourceName; for (Field fld: fields) { resourceName = fld.getName(); try { resourceID = fld.getInt(fld); } catch (IllegalAccessException e) { e.printStackTrace(); } Log.d("RAW FLDINFO","name=" + fld.getName() + " ID=" + String.valueOf(resourceID)); copyResourceImageToImages(resourceID,resourceName, true); } } /** * Copy an image from the raw directory (app/src/main/res/raw directory) to * the Apps data/data/files/images folder * @param resourceID ID of the resource * @param resourceName name of the resource (file name less extension) * @param throw_exception true if an exception should be thrown */ private void copyResourceImageToImages(int resourceID, String resourceName, boolean throw_exception) { String tag = "CPYRSRCTOIMAGES"; InputStream is = getResources().openRawResource(resourceID); File of = new File(images_file.getPath() + File.separator + resourceName + ".jpg"); Log.d(tag,"Initiating Copy of File " + of.getName()); int buffer_size = 1024 * 4; int bytesread = 0; long bytescopied = 0; OutputStream os; byte[] buffer = new byte[buffer_size]; if (!of.exists()) { try { of.createNewFile(); } catch (IOException e) { Log.d(tag,"Error Creating File " + of.getName()); e.printStackTrace(); if (throw_exception) { throw new RuntimeException("Error Creating Output File" + of.getName()); } return; } } try { os = new FileOutputStream(of); } catch (IOException e) { Log.d(tag,"Error Creating OutputStream for File " + of.getName()); e.printStackTrace(); if (throw_exception) { throw new RuntimeException("Error Creating OutputStream for File " + of.getName()); } return; } if (os == null) { throw new RuntimeException("OutputStream not initialised."); } try { while ((bytesread = is.read(buffer)) > 0 ){ try { os.write(buffer, 0, bytesread); } catch (IOException e) { String msg = "Error Writing to Output File " + of.getName() + " Bytes Copied = " + bytescopied; Log.d(tag, msg); e.printStackTrace(); if (throw_exception) { os.close(); of.delete(); throw new RuntimeException(msg); } is.close(); os.close(); of.delete(); return; } bytescopied = bytescopied + bytesread; } }catch (IOException e) { String msg = "Error reading Input File " + resourceName + " Bytes Copied = " + bytescopied; Log.d(tag,"Error Reading Input File " + resourceName); e.printStackTrace(); if (throw_exception) { throw new RuntimeException(msg); } try { is.close(); os.close(); of.delete(); } catch (IOException e2) { e2.printStackTrace(); } } Log.d(tag,"File " + of.getName() +" Copied - Bytes Successfully Copied = " + bytescopied); try { os.flush(); } catch (IOException e) { e.printStackTrace(); } try { os.close(); } catch (IOException e) { e.printStackTrace(); } try { is.close(); } catch (IOException e) { e.printStackTrace(); } } }

    Generally we don't store an image in the sqlite database as it takes a lot of time and is not encouraged but instead we store the image in the private internal storage of our app and we store the name of the image in our sqlite database and thus fetch them accordingly. Databases are not meant for storing images but only text if you try to store images it will lead to large read and write time overheads. You can store the image in internal storage as explained here https://developer.android.com/training/data-storage/files

    来源:https://stackoverflow.com/questions/53954175/how-can-i-insert-image-in-a-sqlite-database

    read more

最新主题

314
Online

339
Users

226.1k
Topics

379.5k
Posts

在线用户

推荐阅读

  • 7
  • 2
  • 1
  • 1
  • 1
  • 1
  • 3
  • 1
  • 2
  • 1
  • 1
  • 1
本站部分内容来自互联网,其发布内容言论不代表本站观点,如果其链接、内容的侵犯您的权益,烦请联系我们,我们将及时予以处理。
Powered by NodeBB | 备案号:宁ICP备15000671号