diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 018d8eb..b61728e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -5,7 +5,12 @@ I welcome requests! Follow these steps to contribute: 1. Find an [issue](https://github.com/AAndyProgram/SCrawler/issues) that needs assistance. 1. Let me know you are working on it by posting a comment on the issue. 1. If you find an error in the code, please provide a link to the file and the line number. -1. If you have a code change suggestion, you can post a replacement code block. I also accept pull requests. +1. If you have a code change suggestion, you can post a replacement code block. + +# How to report a problem +1. Attach a **profile URL** that you cannot download. +1. Attach the **LOG** if it exists. +1. **Attach information to the issue with data copied from SCrawler (click the top right info button in the main window, then the `Environment` button, then the `Copy` button, and paste the copied text into the issue).** # How to build from source 1. Delete the `PersonalUtilities` project from the solution. diff --git a/Changelog.md b/Changelog.md index bba3143..b6c5cf9 100644 --- a/Changelog.md +++ b/Changelog.md @@ -1,3 +1,37 @@ +# 2024.2.25.0 + +*2024-02-25* + +- Added + - A `Feed` button has been added to notifications + - Feed: + - ability to merge multiple special feeds into one + - ability to select all/none media + - ability to add to a special feed(s) with removal from the current one + - the name of the loaded feed is now displayed in the form title + - `Refresh` button now refreshes the loaded feed + - ability to move/copy media + - Scheduler: the ability to move tasks (higher, lower) *(just a view attribute, doesn't affect the scheduler)* + - YouTube (standalone app): add `Open file` to the context menu + - YouTube (standalone app): **the ability to edit each playlist item** + - YouTube (standalone app): **embed thumbnail in the audio/video as cover art** (Settings: `Defaults Audio` - `Embed thumbnail`; `Defaults Video` - `Embed thumbnail (video)`) + - Instagram: the `csrftoken` can now be automatically extracted from cookies + - Instagram: remove `x-ig-www-claim` from settings + - Threads: the `csrftoken` can now be automatically extracted from cookies + - Threads: simplify 500 error when updating tokens + - Facebook: simplify token update errors + - OnlyFans: handle 500 error + - Plugins: added `ReplaceInternalPluginAttribute` attribute + - Other improvements +- Fixed + - Main window: incorrect sorting of profiles and collections + - Standalone downloader: url array form doesn't show scrollbars + - Feed: image rendering bug + - YouTube (standalone app): audio codec does not change when changing audio/video in the video options form + - Instagram: error downloading single post + - TikTok: files with long names aren't downloaded + - Minor bugs + # 2024.1.26.0 *2024-01-26* diff --git a/FAQ.md b/FAQ.md index b7d03d5..14b4e54 100644 --- a/FAQ.md +++ b/FAQ.md @@ -14,8 +14,6 @@ Any other questions I will keep in this file. A: https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies - - ---- #### Q: **Does this program have GUI or CLI.** @@ -41,13 +39,16 @@ A: NO. A: Check your credentials and **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**. If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG. +**You also can join our Discord server**: https://discord.gg/uFNUXvFFmg +
*You can get help faster there!* + **ATTENTION! Issues without URLs will be closed without a response!** ---- #### Q: **I have set credentials but still nothing is downloading** -A: Click the ```Start downloading``` button +A: Click the `Start downloading` button or press `F5` ---- @@ -59,7 +60,7 @@ A: https://github.com/AAndyProgram/SCrawler/releases/latest #### Q: **How to run the program?** -A: Double-click ```SCrawler.exe``` +A: Double-click `SCrawler.exe` ---- @@ -77,19 +78,19 @@ A: The program stored posts IDs in users' folders. For the first time, the progr #### Q: **How to redownload all data** -A: Double-click on the user you want to redownload. In the opened window open folder setting. Delete the files ending with ```_Data.xml``` and ```_Posts.txt```. Restart SCrawler. Download this user again. +A: https://github.com/AAndyProgram/SCrawler/wiki#redownload-user ---- #### Q: **How to remove the label** -A: There is no functionality to remove an individual label. You can open the ```Labels.txt``` file in the program settings folder and delete any label you want. You also can delete this file (```Labels.txt```). In this case, when the program starts, the list of labels list will be updated with only existing labels (from the user data files). +A: There is no functionality to remove an individual label. You can open the `Labels.txt` file in the program settings folder and delete any label you want. You also can delete this file (`Labels.txt`). In this case, when the program starts, the list of labels list will be updated with only existing labels (from the user data files). ---- #### Q: **How to remove a user from the blacklist** -A: Just add that user back to the program. In the dialog box that opens, click on the ```Add and remove from blacklist``` button. +A: Just add that user back to the program. In the dialog box that opens, click on the `Add and remove from blacklist` button. ---- @@ -113,8 +114,8 @@ A: I can only [suggest](#q-you-lost-me-your-program-is-too-complicated) you find #### Q: **Can you add a step-by-step guide or video on how to use the program?** -A: **NO! NEVER!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it. +A: **NO!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it. -**The following video shows how to add credentials:** +**The following video was recorded by a user who loves SCrawler and demonstrates how to add credentials using Instagram as an example:** [![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700) \ No newline at end of file diff --git a/Plugins.md b/Plugins.md index 0f59840..07d3320 100644 --- a/Plugins.md +++ b/Plugins.md @@ -2,6 +2,8 @@ You can create a plugin for any site you want. **To create a plugin, read [this If you've created a plugin, you can create a [new issue](https://github.com/AAndyProgram/SCrawler/issues/new?assignees=&labels=New+Plugin&projects=&template=plugin_add.md&title=%5BNEW+PLUGIN%5D) and I'll add your plugin to the list below. +---- + List of available plugins: Tools: diff --git a/ProgramScreenshots/FeedMoveCopyTo.png b/ProgramScreenshots/FeedMoveCopyTo.png new file mode 100644 index 0000000..c15ceff Binary files /dev/null and b/ProgramScreenshots/FeedMoveCopyTo.png differ diff --git a/ProgramScreenshots/FeedWindow.png b/ProgramScreenshots/FeedWindow.png index 609e90d..6115e93 100644 Binary files a/ProgramScreenshots/FeedWindow.png and b/ProgramScreenshots/FeedWindow.png differ diff --git a/ProgramScreenshots/FeedWindowItemContext.png b/ProgramScreenshots/FeedWindowItemContext.png index 08e29c5..6243ef3 100644 Binary files a/ProgramScreenshots/FeedWindowItemContext.png and b/ProgramScreenshots/FeedWindowItemContext.png differ diff --git a/ProgramScreenshots/FeedWindowSessionContext.png b/ProgramScreenshots/FeedWindowSessionContext.png index 54da5f6..29fec79 100644 Binary files a/ProgramScreenshots/FeedWindowSessionContext.png and b/ProgramScreenshots/FeedWindowSessionContext.png differ diff --git a/ProgramScreenshots/SettingsGlobalFeed.png b/ProgramScreenshots/SettingsGlobalFeed.png index ca6e906..e0f6686 100644 Binary files a/ProgramScreenshots/SettingsGlobalFeed.png and b/ProgramScreenshots/SettingsGlobalFeed.png differ diff --git a/ProgramScreenshots/SettingsSiteInstagram.png b/ProgramScreenshots/SettingsSiteInstagram.png index e6e40a0..ad9865e 100644 Binary files a/ProgramScreenshots/SettingsSiteInstagram.png and b/ProgramScreenshots/SettingsSiteInstagram.png differ diff --git a/ProgramScreenshots/SettingsSiteInstagramData.png b/ProgramScreenshots/SettingsSiteInstagramData.png new file mode 100644 index 0000000..703cb73 Binary files /dev/null and b/ProgramScreenshots/SettingsSiteInstagramData.png differ diff --git a/ProgramScreenshots/SettingsSiteOnlyFans.png b/ProgramScreenshots/SettingsSiteOnlyFans.png index d60a057..9a93e70 100644 Binary files a/ProgramScreenshots/SettingsSiteOnlyFans.png and b/ProgramScreenshots/SettingsSiteOnlyFans.png differ diff --git a/ProgramScreenshots/SettingsSiteRedGifs.png b/ProgramScreenshots/SettingsSiteRedGifs.png index 328405c..ddccbdf 100644 Binary files a/ProgramScreenshots/SettingsSiteRedGifs.png and b/ProgramScreenshots/SettingsSiteRedGifs.png differ diff --git a/ProgramScreenshots/SettingsSiteTikTok.png b/ProgramScreenshots/SettingsSiteTikTok.png index 3694505..d930b34 100644 Binary files a/ProgramScreenshots/SettingsSiteTikTok.png and b/ProgramScreenshots/SettingsSiteTikTok.png differ diff --git a/ProgramScreenshots/SettingsSiteXHamster.png b/ProgramScreenshots/SettingsSiteXHamster.png index 1365fad..59c1f4b 100644 Binary files a/ProgramScreenshots/SettingsSiteXHamster.png and b/ProgramScreenshots/SettingsSiteXHamster.png differ diff --git a/ProgramScreenshots/SettingsSiteYouTube.png b/ProgramScreenshots/SettingsSiteYouTube.png index 18d4891..0ffb850 100644 Binary files a/ProgramScreenshots/SettingsSiteYouTube.png and b/ProgramScreenshots/SettingsSiteYouTube.png differ diff --git a/README.md b/README.md index 2eadf44..b834f0f 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,10 @@ A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, Facebook, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest). **If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)** + +**Join our Discord server**: https://discord.gg/uFNUXvFFmg +
*If you have problems using the program, you can get help faster on our Discord server!* + **Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET @@ -74,12 +78,12 @@ A program to download photo and video from [any site](#supported-sites) (e.g. Yo - **YouTube Music** - **Reddit** - **Twitter** -- **OnlyFans** +- **OnlyFans** *(partial support)*[^1] - **Mastodon** - **Instagram** - **Threads** - **Facebook** -- JustForFans +- JustForFans *(partial support)*[^1] - TikTok - RedGifs - Pinterest @@ -163,6 +167,8 @@ Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) **You can also use the updater included in the release package.** +# [How to report a problem](CONTRIBUTING.md#how-to-report-a-problem) + # [How to build from source](CONTRIBUTING.md#how-to-build-from-source) # [How to make a plugin](https://github.com/AAndyProgram/SCrawler/wiki/Plugins) @@ -220,3 +226,5 @@ Discord server: https://discord.gg/uFNUXvFFmg [Wire](https://account.wire.com/user-profile/?id=93985052-cf2c-4b72-ac75-bbe3231cf544): @andyprogram --> + +[^1]: Partial support means that I don't have personal accounts on paid porn sites because I don't pay for porn. If this site has stopped downloading and you want me to fix it, please be ready to give me access to an account with at least one active subscription. Otherwise, the download from this site will not be fixed. \ No newline at end of file diff --git a/SCrawler.PluginProvider/My Project/AssemblyInfo.vb b/SCrawler.PluginProvider/My Project/AssemblyInfo.vb index c076568..903a261 100644 --- a/SCrawler.PluginProvider/My Project/AssemblyInfo.vb +++ b/SCrawler.PluginProvider/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler.YouTube/My Project/AssemblyInfo.vb b/SCrawler.YouTube/My Project/AssemblyInfo.vb index 22452b9..2e70fe4 100644 --- a/SCrawler.YouTube/My Project/AssemblyInfo.vb +++ b/SCrawler.YouTube/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb b/SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb index 01c2761..3c653f4 100644 --- a/SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb +++ b/SCrawler.YouTubeDownloader/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/SCrawler/API/Instagram/SiteSettings.vb b/SCrawler/API/Instagram/SiteSettings.vb index e9ce04c..3db67ab 100644 --- a/SCrawler/API/Instagram/SiteSettings.vb +++ b/SCrawler/API/Instagram/SiteSettings.vb @@ -249,10 +249,10 @@ Namespace API.Instagram HH_USER_AGENT = New PropertyValue(useragent, GetType(String), Sub(v) ChangeResponserFields(NameOf(HH_USER_AGENT), v)) DownloadTimeline = New PropertyValue(True) - DownloadReels = New PropertyValue(True) + DownloadReels = New PropertyValue(False) DownloadStories = New PropertyValue(True) DownloadStoriesUser = New PropertyValue(True) - DownloadTagged = New PropertyValue(True) + DownloadTagged = New PropertyValue(False) RequestsWaitTimer = New PropertyValue(1000) RequestsWaitTimerProvider = New TimersChecker(100) diff --git a/SCrawler/My Project/AssemblyInfo.vb b/SCrawler/My Project/AssemblyInfo.vb index 3021810..a979ab4 100644 --- a/SCrawler/My Project/AssemblyInfo.vb +++ b/SCrawler/My Project/AssemblyInfo.vb @@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices ' by using the '*' as shown below: ' - - + + diff --git a/Tools/ArchiveSCrawlerUsersDataFiles.bat b/Tools/ArchiveSCrawlerUsersDataFiles.bat index c727306..6e7d187 100644 --- a/Tools/ArchiveSCrawlerUsersDataFiles.bat +++ b/Tools/ArchiveSCrawlerUsersDataFiles.bat @@ -1 +1,7 @@ +REM This script is written for WinRAR. It only archives user data files. Not user content! +REM Replace 'd:\Downloads\SocialNetworks\SCrawlerBackup.rar' with the archive destination. +REM Replace 'd:\Downloads\SocialNetworks\' with the path to your SCrawler data folder. +REM THIS SCRIPT IS NOT SUITABLE FOR 7ZIP OR OTHER ARCHIVING PROGRAMS. +REM But I believe 7Zip also has CLI commands + "C:\Program Files\WinRAR\WinRAR.exe" a -r -ep1 -o+ -ag_YYYYMMDD_HHMMSS -m5 -tl -n*.txt -n*.xml "d:\Downloads\SocialNetworks\SCrawlerBackup.rar" "d:\Downloads\SocialNetworks\" \ No newline at end of file