diff --git a/Home.md b/Home.md index 8e90287..9430772 100644 --- a/Home.md +++ b/Home.md @@ -79,9 +79,11 @@ This button looks like a bookmark. When you click on this button, a form for dow - ```Favorite``` - only show users marked as favorite. - ```Deleted``` - only show users whose profiles have been deleted - ```Suspended``` - only show users whose profiles have been suspended - - ```Labels``` - show users labeled with one or more selected label (click ```Select labels``` to select labels). - - ```No Labels``` - only show users without a label. - - ```Select labels``` - select one or more labels to display. + - ```Labels``` - show users labeled with one or more selected label + - ```No Labels``` - only show users without a label + - ```Excluded labels``` - exclude from displaying users with the selected labels (this is suitable for situations where you want to hide a user but don't want to delete) + - ```Ignore excluded labels``` - Ignore excluding labels. This allows you to show users that should be excluded from display without resetting the current selection of excluded labels. + - ```Show groups instead of labels``` - show sites groups instead of labels (when filtering users by labels) - ```Limit dates``` - set lowest date to filter out users who have not been downloaded since that date ![Main window context](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/MainWindow2.png) diff --git a/Settings.md b/Settings.md index d3ca8fe..35e207c 100644 --- a/Settings.md +++ b/Settings.md @@ -1,46 +1,57 @@ # Program settings -- Basis - - ```Data path``` - this is the root path where the data will be placed. - - ```Large/Small image size``` - the size of the user icons. - - ```Collections folder``` - just the name of the collections folder. - - ```Maximum download tasks of users``` - the number of simultaneously downloading profiles. - - ```Maximum download tasks of channels``` - the number of simultaneously downloading channels. - - ```Check new version at start``` - - ```Imgur Client ID``` - Imgur client ID to Bypass NSFW protection and download galleries - - ```Change file names``` - name files by name (not by original names) - - Options: - - ```Replace file name by date``` - the file name will be replaced with the date of the file was posted - - ```Add date/time to file name``` - the date will be appended to the file name - - Parameters: - - ```Date``` - append date to file name - - ```Time``` - append time to file name - - Date positions ```Start/End``` - date and/or time will be appended to the end or beginning of the file name - - ```Exit confirm``` - ask for confirmation before closing the program - - ```Close to tray``` - close program to system tray - - ```Show notifications``` - show notifications when download is complete - - ```Fast profiles loading``` - fast loading profiles in the main window. **Be careful with this setting. Fast loading leads to the highest CPU usage.** - - ```Folder cmd``` - the [command](#folder-command) to open a folder - - ```Delete data to recycle bin``` - delete data to recycle bin or permanent - - ```Script``` - [script](#how-to-use-the-script) to be executed after the user download is complete. If the checkbox is checked, new users will be created with the ```Use script``` option. +## Basis +- ```Data path``` - this is the root path where the data will be placed. +- ```Large/Small image size``` - the size of the user icons. +- ```Collections folder``` - just the name of the collections folder. +- ```Maximum download tasks of users``` - the number of simultaneously downloading profiles. +- ```Maximum download tasks of channels``` - the number of simultaneously downloading channels. +- ```Check new version at start``` +- ```Imgur Client ID``` - Imgur client ID to Bypass NSFW protection and download galleries +- ```Show groups``` - Show user site groups +- ```Use user grouping``` - Group users by groups and/or labels. If disabled, users will not be grouped by site or label. - ![Basis settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsBasis.png) + ![Basis settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalBasis.png) -- Defaults - - ```Separate video folder``` - this means that video files will be placed in a separate folder in the user's folder to store video files separately from images. - - ```Temporary``` - this parameter specifies how users will be created by default in the user creation form. - - ```Download images/videos``` - defaults for creating new users - - ```Update user description every time``` - this means that the user description (if implemented) will be checked when the user is parsed. If the current user description does not contain a new description, then a new one will be added via a new line. +## Behavior - ![Default settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingDefaults.png) +- ```Exit confirm``` - ask for confirmation before closing the program +- ```Close to tray``` - close program to system tray +- ```Show notifications``` - show notifications when download is complete +- ```Fast profiles loading``` - fast loading profiles in the main window. **Be careful with this setting. Fast loading leads to the highest CPU usage.** +- ```Delete data to recycle bin``` - delete data to recycle bin or permanent +- ```Folder cmd``` - the [command](#folder-command) to open a folder -- Channels - - ```Channels rows/columns``` - how many rows and columns will be displayed in the channels form. For example: rows = 2; columns = 5. In this case, 10 images will be placed on one page in two rows of 5 columns. **Please don't set too high value.** - - ```Download limit for channel user``` - the amount of media will be downloaded if the user added from the channel. - - ```Copy channel user image``` - the image posted by user in the channel will be copied to the user folder when user will be created. - - ```Create temporary users``` - users will be created marked 'Temporary' when created from channel. + ![Basis settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalBehavior.png) - ![Channels settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsChannels.png) +## Defaults +- ```Separate video folder``` - this means that video files will be placed in a separate folder in the user's folder to store video files separately from images. +- ```Temporary``` - this parameter specifies how users will be created by default in the user creation form. +- ```Download images/videos``` - defaults for creating new users + + ![Default settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalDefaults.png) + +## Downloading +- ```Update user description every time``` - this means that the user description (if implemented) will be checked when the user is parsed. If the current user description does not contain a new description, then a new one will be added via a new line. +- ```Change file names``` - name files by name (not by original names) + - Options: + - ```Replace file name by date``` - the file name will be replaced with the date of the file was posted + - ```Add date/time to file name``` - the date will be appended to the file name + - Parameters: + - ```Date``` - append date to file name + - ```Time``` - append time to file name + - Date positions ```Start/End``` - date and/or time will be appended to the end or beginning of the file name +- ```Script``` - [script](#how-to-use-the-script) to be executed after the user download is complete. If the checkbox is checked, new users will be created with the ```Use script``` option. + + ![Default settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalDownloading.png) + +## Channels +- ```Channels rows/columns``` - how many rows and columns will be displayed in the channels form. For example: rows = 2; columns = 5. In this case, 10 images will be placed on one page in two rows of 5 columns. **Please don't set too high value.** +- ```Download limit for channel user``` - the amount of media will be downloaded if the user added from the channel. +- ```Copy channel user image``` - the image posted by user in the channel will be copied to the user folder when user will be created. +- ```Create temporary users``` - users will be created marked 'Temporary' when created from channel. + + ![Channels settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsGlobalChannels.png) # Site settings @@ -55,7 +66,7 @@ The Reddit parser can parse data without cookies, but you can add it if you like - ```Saved posts path``` - if you wish, you can specify a special path for saved posts. Leave it blank to use the default path. - ```Saved posts user``` - your personal Reddit username to download your saved posts (this feature requires cookies) - ![Reddit settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsReddit.png) + ![Reddit settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteReddit.png) ## Reddit user settings @@ -72,7 +83,7 @@ The Reddit parser can parse data without cookies, but you can add it if you like - ```Authorization``` - Bearer token. Must start with ```Bearer ``` word followed by a space. - ```Token``` - in this field you need to put ```x-csrf-token```. - ![Twitter settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsTwitter.png) + ![Twitter settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteTwitter.png) ### How to find Twitter tokens @@ -105,7 +116,7 @@ The Reddit parser can parse data without cookies, but you can add it if you like - ```Get stories``` - ```Get tagged photos``` - ![Instagram settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsInstagram.png) + ![Instagram settings](https://github.com/AAndyProgram/SCrawler/blob/main/ProgramScreenshots/SettingsSiteInstagram.png) ### How to find Instagram Hash 2 @@ -128,10 +139,6 @@ The Reddit parser can parse data without cookies, but you can add it if you like ### Instagram limits Instagram API is requests limited. For one request, the program receive only 50 posts. Before catching error 429, the program can process 200 requests. I reduced this to 195 requests and set a timer to wait for the next request after. - -~~Be careful with ```Ready for download``` option. I recommended parsing one new profile every 5-10 minutes. I'm still thinking about how to make Instagram parsing algorithm smoother.~~ - -~~I think you can get new posts for several profiles that have already parsed using the ```Download all``` operation without the "Too many requests" error. But if the program catches an error (429), the timeout information will be added to the log.~~ ## RedGifs