geonu/SCrawler
1
0
Fork 0
mirror of https://github.com/AAndyProgram/SCrawler.git synced 2026-03-14 15:52:18 +00:00
Code Issues Packages Projects Releases Wiki Activity

2022.9.24.0

Andy
2022-09-24 20:30:44 +03:00
parent ec9e68ec83
commit 63db58d378
2 changed files with 29 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
Home.md

@@ -188,6 +188,7 @@ Double-click the user to open the content folder.
- ```Download data to the date``` - download this user/collection data up to selected date.
- ```Edit user information``` - open a form to change user parameters.
- ```Delete user/collection``` - delete one or more selected users/collections.
- ```Copy data to another folder``` - allows you to copy user data to another folder.
- ```Change favorite``` - if the user was marked as favorite, he becomes a regular; otherwise, he becomes the favorite.
- ```Change temporary``` - if the user was marked as temporary, he becomes a regular; otherwise, he becomes the temporary.
- ```Change ready for download``` - download this user in the operation ```Download all```.

43
Settings.md

@@ -48,6 +48,8 @@
- Date positions ```Start/End``` - date and/or time will be appended to the end or beginning of the file name
- ```Script``` - [script](#how-to-use-the-script) to be executed after the user download is complete. If the checkbox is checked, new users will be created with the ```Use script``` option.
- ```After download cmd``` - this [command](#scrawler-script-text-examples) will be executed after all downloads are completed
- ```Add missing information to log``` - Text will be added to the log stating that the [missing posts](https://github.com/AAndyProgram/SCrawler/wiki/#missing-posts) exist.
- ```Add missing errors to log``` - Each error that prevents SCrawler from downloading a file will be added to the log.
![Default settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalDownloading.png)
@@ -58,11 +60,13 @@
- ```Create temporary users``` - users will be created marked 'Temporary' when created from channel.
![Channels settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalChannels.png)
## Feed
- ```Feed rows/columns``` - how many rows and columns will be displayed in the feed form
- ```Endless feed``` - go to the next page when you reach the end
- ```Add the session number to the post title``` - The session number will be displayed to the left of the post title. A session is every time when you (or [AutoDownloader](#automation)) download data.
- ```Add the date to the post title``` - The download date and time will be displayed to the right of the post title.
![Feed settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalFeed.png)
@@ -70,19 +74,20 @@
## Reddit
You need to configure Reddit path where Reddit users' data will be placed. Default value: ```Data path``` + "Reddit".
The Reddit parser can parse data without cookies, but you can add it if you like.
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Path``` - where Reddit users' data will be placed.
- ```Saved posts path``` - if you wish, you can specify a special path for saved posts. Leave it blank to use the default path. Saved Twitter posts are posts that you have bookmarked.
- ```Cookies``` add cookies from your browser (click the pencil button to open the cookies editor).
- ```Download site data``` - You can disable downloading data from the site if you need it. If disabled, this site's data will not be downloaded.
- ```Get user media only``` - the default for creating new users
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Saved posts path``` - if you wish, you can specify a special path for saved posts. Leave it blank to use the default path.
- ```Saved posts user``` - your personal Reddit username to download your saved posts (this feature requires cookies)
- ```Use M3U8``` - If you are using an x86 program, you will not be able to use the M3U8 download. In other cases, this setting controls how to download videos hosted on Reddit. There are two ways to download: the mp4 file as is, but without an audio track, and through an m3u8 playlist that contains an audio track.
![Reddit settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteReddit.png)
### Reddit user settings
- ```View``` - works the same as Reddit view modes (```new```, ```hot```, ```top```)
@@ -92,15 +97,16 @@ The Reddit parser can parse data without cookies, but you can add it if you like
## Twitter
- ```Path``` - where Twitter users' data will be placed.
- ```Saved posts path``` - if you wish, you can specify a special path for saved posts. Leave it blank to use the default path. Saved Twitter posts are posts that you have bookmarked.
- ```Cookies``` add cookies from your browser (click the pencil button to open the cookies editor).
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Download site data``` - You can disable downloading data from the site if you need it. If disabled, this site's data will not be downloaded.
- ```Get user media only``` - the default for creating new users
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Authorization``` - Bearer token. Must start with ```Bearer ``` word followed by a space.
- ```Token``` - in this field you need to put ```x-csrf-token```.
![Twitter settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteTwitter.png)
### How to find Twitter tokens
1. Open Google Chrome, Edge or FireFox.
@@ -113,19 +119,19 @@ The Reddit parser can parse data without cookies, but you can add it if you like
- ```authorization: Bearer .......``` - this is the authorization token. You need to copy the text from starting with the word ```Bearer``` to the end of the line and paste it into the ```Authorization``` field on the Twitter settings form.
- ```x-csrf-token: ...............``` - this is csrf token. You need to copy all the text after the words ```x-csrf-token:``` and paste it into the ```Token``` field on the Twitter settings form
## Instagram
- ```Path``` - where Instagram users' data will be placed.
- ```Cookies``` add cookies from your browser (click the pencil button to open the cookies editor).
- ```Saved posts path``` - if you wish, you can specify a special path for saved posts. Leave it blank to use the default path.
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Cookies``` add cookies from your browser (click the pencil button to open the cookies editor).
- ```Download site data``` - You can disable downloading data from the site if you need it. If disabled, this site's data will not be downloaded.
- ```Get user media only``` - the default for creating new users
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- Authorization
- ```Hash``` - **\[Required\]** in this field you need to put the hash of the Instagram session ([how to find](#how-to-find-instagram-hash)).
- ```Hash 2``` - **\[For saved Instagram posts only\]** in this field you need to put the [hash of the Instagram session for saved posts](#how-to-find-instagram-hash).
- ```x-csrftoken``` - **\[Required\]** read [here](#how-to-find-instagram-authorization-headers) how to find them
- ```x-csrftoken``` - **\[Required\]** read [here](#how-to-find-instagram-authorization-headers) how to find them
- ```x-ig-app-id```, ```ix-ig-www-claim``` - **\[Required\]** read [here](#how-to-find-instagram-authorization-headers) how to find them
- ```Saved posts user``` - your personal Instagram username to download your saved posts (this feature requires cookies and **InstaHash 2**)
- Other parameters
@@ -134,8 +140,8 @@ The Reddit parser can parse data without cookies, but you can add it if you like
- ```Post limit timer``` - this is the time value (in milliseconds) the program will wait before processing the next request after 195 requests (**it is highly recommended not to change this default value**)
- ```Get stories```
- ```Get tagged photos```
- ```Tagged notify limit``` - Limit of new tagged posts when you receive a notification (read more [here](#instagram-tagged-posts-limit))
- ```Tagged notify limit``` - Limit of new tagged posts when you receive a notification (read more [here](#instagram-tagged-posts-limit))
![Instagram settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteInstagram.png)
### How to find Instagram Hash
@@ -175,10 +181,17 @@ When the number of tagged posts exceeds the tagged posts limit, you will be aske
- ```Cancel``` - Cancel tagged posts download operation
- ```Disable and cancel``` - Disable downloading tagged data and cancel downloading tagged data
- ```Cancel``` - Cancel tagged posts download operation
## RedGifs
```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
- ```Path``` - where RedGifs users' data will be placed.
- ```Saved posts path``` - *unused*
- ```Cookies``` add cookies from your browser (click the pencil button to open the cookies editor).
- ```Download site data``` - You can disable downloading data from the site if you need it. If disabled, this site's data will not be downloaded.
- ```Get user media only``` - the default for creating new users
- ```Temporary```, ```Download images/videos``` - personal site settings by default for creating new users. Means the same as the parameters of the same name on the ```Defaults``` tab. If the checkbox is in an intermediate state, then the same name setting of the ```Defaults``` tab will be used instead. Otherwise, this setting will be applied.
![RedGifs settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteRedGifs.png)
# Download groups