mirror of https://github.com/AAndyProgram/SCrawler.git synced 2026-03-14 15:52:18 +00:00

Files

Andy b2a9b22478 2023.4.28.0

Plugins
IPluginContentProvider: added DownloadSingleObject function; added tokens to GetMedia and Download functions; removed GetSpecialData function
Add IDownloadableMedia interface
Removed 'Channel' option from all functions and enums
ISiteSettings: added GetSingleMediaInstance function
ExchangeOptions: removed 'IsChannel'
UserMediaTypes: added Audio and AudioPre enums
IUserMedia, PluginUserMedia: changed ContentType and DownloadState from integers to their enums

SCrawler
Add YouTube standalone downloader
Add gallery-dl & yt-dlp support
Remove 'UserInfo' requirement from 'ProfilesSaved'
Update 'SiteSettingsBase' to use domains and Netscape cookies
UserDataBase: remove channels; remove old 'Merge' const; standardize SavedPosts file naming; move 'ValidateMD5' function from Twitter to UserDataBase to use it in other UserData classes; add 'DownloadSingleObject' environment for single posts; add validating file extension for m3u8 during download; add reindex of video file during download

Rewritten DomainsContainer
Create a universal settings form and PSettingsArttribute
Gfycat, Imgur: turn these classes into IUserData to download a single object

All plugins: update 'GetInstance' function for saved posts; update domains where implemented; remove 'OptionForm' where it exists; update options where they exist; update unix date providers; reconfigure channels where they exist

LPSG: fix attachments; update converters and regex
Add sites: ThisVid, Mastodon, Pinterest, YouTube, YouTube music
Reddit: standardize container parsing for all data types; new channel environment; fix 'ReparseMissing' function; redirect data downloading to the base download function, saved crossposts support
Twitter: fixed gif path bug; fixed downloading saved posts
PornHub: hide unnecessary errors; photo galleries bug
RedGifs: add 'UserAgent' option

Added icons to download progress

Rename some objects
Completely redesigned standalone downloader form and rewritten its environment
WebClient2: update to use tokens

Labels: update label form (save labels to file only when OK button is clicked); change removing labels.txt from recycle bin to permanent; disable storing label 'NoParsedUser'

UserCreatorForm: remove the 'Channel' checkbox and related functions; ability to extract the user's URL from the buffer and apply parameters if found
Remove temporary 'EncryptCookies' module

MainFrame: added simplified way to create new users (Ctrl+Insert to create a new user with default parameters from clipboard URL); removed SCrawler command line argument "-v" (remove the ability to run SCrawler as video downloader)
PropertyValueHost: update for option forms compatibility
SettingsHost: removed 'GetSpecialData' fork; added 'GetSingleMediaInstance' fork
UserDataHost: update functions with tokens; update events; add 'DownloadSingleObject' function
Settings: add the ability to get environment from 4 destinations; add the ability to set the program environment manually; add CMDEncoding; add cache; remove the old function 'RemoveUnusedPlugins'; add 'STDownloader' properties; add YT compatibility; add new notification options; add deleting user settings file when 'SettingsCLS.Dispose()' if where are no users in SCrawler
UserFinder: remove old 'Merge' const; remove channel option
UserInfo: remove channel option

2023-04-28 10:13:46 +03:00

2.2 KiB

Raw Blame History

Contributor's Guide

I welcome requests! Follow these steps to contribute:

Find an issue that needs assistance.
Let me know you are working on it by posting a comment on the issue.
If you find an error in the code, please provide a link to the file and the line number.
If you have a code change suggestion, you can post a replacement code block. I also accept pull requests.

How to build from source

Delete the PersonalUtilities project from the solution.
Delete the PersonalUtilities.Notifications project from the solution.
The following libraries must be added to project references with the 'Copy to output folder' option:
- PersonalUtilities.dll
- PersonalUtilities.Notifications.dll
- Microsoft.Toolkit.Uwp.Notifications.dll
- System.ValueTuple.dll
Import PersonalUtilities.Functions for the whole project.

Always use the correct libraries. You must download libraries from the same release date as the code commit date.

How to request a new site

I'm currently not accepting requests to develop new sites.

Check issues (open and closed) and discussions to find your issue. Perhaps I have already answered your request.
If you don't find anything, create a new issue with your request. I usually reply as soon as possible (within the next few hours).

Requirements for new site requests

Attention! I'll add a new site only if I'm interested. I also have a life, and any development takes time.

Post a link to the site's API
Post request URLs without OAuth authentication
Post a complete cURL request which provides the required information (JSON is better)

I don't use OAuth authentication in my application, so if it's not too hard to make a new parsing algorithm without OAuth authorization, I can start developing it in the coming days. Otherwise, I need time to figure out how to do it.

If I'm interested in a site you want to add, it may be added in future releases.

Sites I will never develop

Facebook
Tumblr