mirror of
https://github.com/AAndyProgram/SCrawler.git
synced 2026-03-14 15:52:18 +00:00
2022.10.18.0
Moved UserMedia xml initialization to the structure itself Added download with feed skip Added silent mode (temporary disabling notifications) Added additional Instagram protection Excluding users whose profiles do not exist from downloading with groups and AutoDownloader Feed: delete file bugs; reorder data after file deletion; video playback bugs SiteSettingsForm: enable 'OK' button when editing cookies Fixed collection users ban Settings: disabling ffmpeg missing notification; advanced notification management Added 'ToolStripKeyMenuItem' control Plugins: deprecated XVIDEOS and LPSG plugin libraries; moved them to SCrawler. Updated license PluginProvider: added 'BeginEdit' and 'EndEdit' function to ISiteSettings; changed GetSpecialData (ISiteSettings) return type to IEnumerable PluginsEnvironment: removed 'IsMyClass' attribute MainFrame: grouped all download buttons into one menu; reorganized code; removed 'F2' hotkey AutoDownloader: added advanced pause options; added buttons to tray icon and AutoDownloader form MissingPosts: finished; activated functions that were disabled; added download functions to UserData classes UserDataBase: ability to use responser; ability to download m3u8; extended 'DownloadingException' with optional argument 'EObj'; user index in collection (button tag) changed to user instance; extended information with user labels; updated 'ProcessException' function Replaced download buttons with 'KeyClick' control Replaced FDatePickerForm with my library's form Collections: Deleting multiple collections - disabled confirmation; ban each user in collection
This commit is contained in:
Andy
70
README.md
70
README.md
@@ -6,7 +6,7 @@
|
||||
[](https://github.com/AAndyProgram/SCrawler/wiki)
|
||||
[](HowToSupport.md)
|
||||
|
||||
A program to download photo and video from [any site](#supported-sites) (e.g. Reddit, Twitter, Instagram).
|
||||
A program to download photo and video from [any site](#supported-sites) (e.g. Reddit, Twitter, Instagram, TikTok, RedGifs, XVIDEOS, LPSG).
|
||||
|
||||
**If you like SCrawler, please like the program on [this site]( https://alternativeto.net/software/scrawler/about/)**
|
||||
|
||||
@@ -21,14 +21,11 @@ Do you like this program? Consider adding to my coffee fund by making a donation
|
||||
|
||||
# What can program do:
|
||||
- Download pictures and videos from users' profiles and subreddits:
|
||||
- Reddit images;
|
||||
- Reddit galleries of images;
|
||||
- Reddit videos (downloading Reddit hosted video is going through ffmpeg (**ffmpeg only works with the x64 program**));
|
||||
- Reddit images, galleries of images, videos (downloading Reddit hosted video is going through ffmpeg (**ffmpeg only works with the x64 program**));
|
||||
- Redgifs videos (https://www.redgifs.com/);
|
||||
- Twitter images and videos;
|
||||
- Instagram images and videos;
|
||||
- Instagram tagged posts;
|
||||
- Instagram stories;
|
||||
- Instagram images and videos, tagged posts, stories;
|
||||
- TikTok videos ([limited](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok-limits));
|
||||
- Imgur images, galleries and videos;
|
||||
- Gfycat videos;
|
||||
- [Other](#supported-sites) supported sites
|
||||
@@ -55,6 +52,7 @@ Do you like this program? Consider adding to my coffee fund by making a donation
|
||||
- **Reddit**
|
||||
- **Twitter**
|
||||
- **Instagram**
|
||||
- **TikTok** ([limited](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok-limits))
|
||||
- RedGifs
|
||||
- Imgur
|
||||
- Gfycat
|
||||
@@ -62,7 +60,9 @@ Do you like this program? Consider adding to my coffee fund by making a donation
|
||||
- XVIDEOS
|
||||
- [Other sites](Plugins.md)
|
||||
|
||||
# How does it works:
|
||||
**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
|
||||
|
||||
# How it works
|
||||
|
||||
First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.
|
||||
|
||||
@@ -74,8 +74,6 @@ The program parses all user posts, obtain MD5 images hash and compares them wit
|
||||
|
||||
The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
|
||||
|
||||
You can read about Instagram restrictions [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-limits)
|
||||
|
||||
## How to request a new site
|
||||
|
||||
Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about
|
||||
@@ -83,12 +81,33 @@ Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about
|
||||
# Requirements
|
||||
|
||||
- Windows 10, 11 with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
|
||||
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for Twitter (if you want to download data from Twitter)
|
||||
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) for Instagram (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for saved Instagram posts, Instagram [stories authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-stories-authorization-headers) for Stories and Tagged data
|
||||
- ffmpeg library for downloading videos hosted on Reddit (you can download it from the [official repo](https://github.com/GyanD/codexffmpeg/releases/tag/2021-01-12-git-ca21cb1e36) or [from my first release](https://github.com/AAndyProgram/SCrawler/releases/download/1.0.0.0/ffmpeg.zip)). **ffmpeg only works with the x64 version of the program.**
|
||||
- **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
|
||||
|
||||
# Guide
|
||||
|
||||
- [Main window](https://github.com/AAndyProgram/SCrawler/wiki)
|
||||
- [Users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
|
||||
- [Add/Edit/Delete users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
|
||||
- [Collections](https://github.com/AAndyProgram/SCrawler/wiki#collections)
|
||||
- [User operations](https://github.com/AAndyProgram/SCrawler/wiki#context-menu)
|
||||
- [User labels](https://github.com/AAndyProgram/SCrawler/wiki/Users#labels)
|
||||
- **[DOWNLOAD](https://github.com/AAndyProgram/SCrawler/wiki#download)**
|
||||
- [Automation](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation)
|
||||
- [Download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
|
||||
- [Downloading information](https://github.com/AAndyProgram/SCrawler/wiki#info)
|
||||
- [Reddit channels](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
|
||||
- [Saved posts](https://github.com/AAndyProgram/SCrawler/wiki#saved-posts)
|
||||
- [View modes, filters](https://github.com/AAndyProgram/SCrawler/wiki#view)
|
||||
- **[SETTINGS](https://github.com/AAndyProgram/SCrawler/wiki/Settings)**
|
||||
- **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
|
||||
- [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
|
||||
- [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
|
||||
- [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
|
||||
- [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
|
||||
- [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)
|
||||
- [XVIDEOS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xvideos)
|
||||
- [LPSG](https://github.com/AAndyProgram/SCrawler/wiki/Settings#lpsg)
|
||||
|
||||
**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
|
||||
|
||||
# Installation
|
||||
@@ -117,22 +136,9 @@ Read more about how to support the program [here](HowToSupport.md).
|
||||
|
||||
The program has an intuitive interface.
|
||||
|
||||
You need to set up authorization for Twitter and Instagram:
|
||||
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for **Twitter** (if you want to download data from Twitter)
|
||||
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies), [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) and [authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-authorization-headers) for **Instagram** (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for **saved Instagram posts**
|
||||
**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
|
||||
|
||||
Just add a user profile and **click the ```Start downloading``` button**.
|
||||
|
||||
You can add users by patterns:
|
||||
- https://www.instagram.com/SomeUserName
|
||||
- https://twitter.com/SomeUserName
|
||||
- https://reddit.com/user/SomeUserName
|
||||
- https://reddit.com/r/SomeSubredditName
|
||||
- https://www.redgifs.com/users/SomeUserName
|
||||
- u/SomeUserName
|
||||
- r/SomeSubredditName
|
||||
- SomeUserName (in this case, you need to select the user's site)
|
||||
- SomeSubredditName
|
||||
Just add a user profile and **click the ```Download``` button**.
|
||||
|
||||
Read more about adding users and subreddits [here](https://github.com/AAndyProgram/SCrawler/wiki/Users)
|
||||
|
||||
@@ -144,10 +150,4 @@ Create a shortcut for the program. Open shortcut properties. In the ```Shortcut`
|
||||
|
||||
Example: ```D:\Programs\SCrawler\SCrawler.exe v```
|
||||
|
||||
|
||||
|
||||
# Contact me
|
||||
|
||||
[](https://matrix.to/#/@andyprogram:matrix.org)
|
||||
|
||||
[](https://discordapp.com/users/1012768226679206009) AndyProgram#3804
|
||||
|
||||
Reference in New Issue
Block a user