Compare commits

...

128 Commits

Author SHA1 Message Date
Andy
5b5857e31d 2024.4.14.0
Delete old notes and comments
API.Facebook: add app-id extraction from page; remove app-id from site requirements; update tokens parsing; update tokens regex
API.Instagram: add default function to parse tokens
2024-04-14 07:44:05 +03:00
Andy
46372ec9fb 2024.4.13.0
YT
Add subtitles to information about downloaded files

PluginProvider
IPluginContentProvider: add 'ResetHistoryData' function

SCrawler
UserDataBase: call 'UpdateUsersList' when 'UpdateUserInformation' with argument 'OnlyDiff'; implement 'ResetHistoryData' function; set 'LastUpdated' to null and 'UpdateUserInformation' when erasing history data
API.Instagram: set 'FirstLoadingDone' to false when erasing history data; fix broken saved posts downloading
API.TikTok: set 'LastDownloadDate' to null when erasing history data
API.YouTube: set last download dates to null when erasing history data
GroupUsersViewer: add the number of users, object type and object name to the form title; add 'F1' to help hint; add '(Alt+)F3' to edit user
AutoDownloaderEditorForm, SchedulerEditorForm, GroupEditorForm, GroupListForm: update to 'GroupUsersViewer'
MainFrame: make 'EditSelectedUser' friend
UserDataHost: implement 'ResetHistoryData' function
SettingsCLS: add 'OnlyDiff' to the 'UpdateUsersList' function
UserInfo: add 'ExactEquals' shared function
2024-04-13 10:10:27 +03:00
Andy
7296fda977 2024.4.10.0 2024-04-10 06:25:39 +03:00
Andy
5f90bf6a99 2024.4.8.0
YT
MusicPlaylistsForm, VideoOptionsForm: add audio bitrate option
MediaItem: update type icon; update confirmation dialog for deleting non-single object
Track: update extension
PlayList: update 'ToString' information for 'MediaItem'
YouTubeMediaContainerBase: add size recalculation; add audio bitrate change; embed thumbnail in the extracted 'mp3' as cover art; update 'DownloadCommand' function; include elements' files in XML for non-single items; update 'Delete' function; update files handling; include generated playlists and cover file in the file list
YouTubeSettings: add properties 'DefaultAudioEmbedThumbnail_ExtractedFiles', 'DefaultAudioBitrate', 'DefaultAudioBitrate_crf'
Exclude 'drc' from parsing results
Fix incorrect file reference when the yt-dlp.exe has a different name

SCrawler
Base.Declarations: hide 'TokenRefreshIntervalProvider' error
Base.DeclaredNames: remove 'Header_FB_FRIENDLY_NAME' const (use 'API.Instagram.UserData.GQL')
Base.M3U8Base: add 'SkipBroken' argument
Base.UserDataBase: add size recalculation (STD)
Base.SiteSettingsBase: add 'SettingsVersion' property
TDownloader: delete 'RenameOldFileNames' function
SiteEditorForm: remove begin/end update of global settings when updating
MainFrame: update 'BTT_DOWN_SPEC' tooltip
SettingsHostCollection, SettingsHost: move site settings to a personal setting file (delete these settings from the global settings file)
DownloadGroupCollection: remove data update during initialization and reindexing
SettingsCLS: add 'SettingsVersion' property
Add hidden controls
API.JustForFans: change m3u8 parsing and downloading algo; remove 'CancellationToken' from m3u8 (replace with 'IThrower')
API.Facebook: add option 'RequestsWaitTimer_Any'; add internal option 'DownloadData_Impl'; update GDL names and tokens references; add wait timers
API.Threads: add option 'RequestsWaitTimer_Any'; add internal option 'DownloadData_Impl'; update GDL names and tokens references; add wait timers
API.Instagram: ADD 'GDL' SUPPORT; add 'UpdateWwwClaim' to 'Declarations.UpdateResponser' and 'UserData'; add additional 'HH_IG_WWW_CLAIM' properties; add 'RequestsWaitTimer_Any' property; add tooltips for timer controls; update 'LastRequests' environment; update information about requests on the label in the settings form; update reels downloading function
2024-04-09 21:39:09 +03:00
Andy
718eccc3c3 2024.3.31.0
Plugins
ISiteSettings: add 'CMDEncoding', 'EnvironmentPrograms' properties; add 'EnvironmentProgramsUpdated' function

SCrawler
GlobalSettingsForm: add the ability to find the program environment
SettingsCLS: update 'ProgramFile' class; add 'EnvironmentProgramsList' property and 'UpdateEnvironmentPrograms' function
Add the ability to display group users
Add 'GroupUsersViewer'
2024-03-31 13:18:20 +03:00
Andy
efa09fb457 2024.3.30.0
UPDATE DOWNLOAD GROUP ENVIRONMENT

Add the ability to filter users who have been (not)downloaded in the last x days.
DownloadedInfoForm: fix possible bug
Feed: fix scrolling bug
IUserData, UserDataBase, UserDataBind: remove 'FitToAddParams'
UserDataBase: update 'GetLVIGroup' function; wrong decision to set 'LastUpdated' date
AutoDownloader: remove 'All' and 'Default' options
SettingsCLS: refactoring the code and XML file
2024-03-30 15:42:30 +03:00
Andy
b252d32a7e 2024.3.26.0
API.Instagram: extract image from video
API.Reddit: add 'TryImage' bypass
Feed: add hotkeys: 'Home', 'End', 'Up', 'Page Up', 'Down', 'Page Down'; fix form deactivating; add ability to save/load view
MainFrame: update the background picture if it has changed
2024-03-26 14:13:01 +03:00
Andy
34cd510507 2024.3.24.0
YT
YouTubeMediaContainerBase: add item index to playlist row only if it is > 0

SCrawler
API.Instagram: handle 'JsonNull' for saved posts
GlobalSettingsForm: move the 'GroupUser' option to MainFrame
MainFrame: add 'GroupUser' option and 'ViewFilter' options
SettingsHostCollection: update the 'Silent' parameter for single instance
2024-03-24 02:56:08 +03:00
Andy
f5dd791941 2024.3.22.0
TDownloader: exclude the last 3 days from deleting session files
UserDownloadQueueForm: fix bug
2024-03-22 01:28:49 +03:00
Andy
2a2c12c651 2024.3.19.0
API.XHamster: some videos are missing when downloading creators; user videos aren't downloading
2024-03-19 01:09:54 +03:00
Andy
2bacc17ac4 2024.3.17.0
API.OnlyFans: add stories download
API.PornHub: fix list trim
2024-03-18 02:27:01 +03:00
Andy
7a68067d77 2024.3.15.0
PluginProvider
Add 'Checked' property
Add 'PropertyValueEventArgs'

SCrawler
Add default headers
Add predefined colors to the color picker
API.Instagram: automatically reset download options after updating credentials
2024-03-15 21:38:55 +03:00
Andy
d1eacc2db2 2024.3.13.0
YT
Add the ability to add downloaded item(s) to a playlist
Add the ability to create a playlist from downloaded music

SCrawler
Fix resetting of 'Available' indexes when the exiting a job
Allow thumbnail to be saved with file
API.TikTok: add more settings for standalone downloader; fix files with long names aren't downloaded (STD)
Feed: add moved/copied files to message box
2024-03-13 23:34:00 +03:00
Andy
3f38803643 2024.3.4.0
YT
Add the ability to add downloaded item(s) to a playlist
Add the ability to create a playlist from downloaded music

SCrawler
Fix resetting of 'Available' indexes when the exiting a job
2024-03-04 21:09:38 +03:00
Andy
09760a6926 2024.2.25.0 2024-02-25 11:31:33 +03:00
Andy
75039ac4d2 2024.2.24.0
Feed: improve move/copy
2024-02-24 12:24:05 +03:00
Andy
03e3a07947 2024.2.22.1
Feed: add the ability to update file location when moving media
Automation: make the automation file relative
2024-02-22 19:58:46 +03:00
Andy
d283d9c9f9 2024.2.22.0
Feed: add the ability to move/copy media
2024-02-22 13:40:04 +03:00
Andy
b9100bd3c1 2024.2.21.0
API.UserDataBase: incorrect comparison when 'Other' is 'UserDataBind'
DownloadFeedForm: add saved session name to feed title; remove menu hiding when session saving is disabled; remove menu tooltip
GlobalSettingsForm: update 'FeedParametersChanged'
2024-02-21 09:19:40 +03:00
Andy
0ea2156ada 2024.2.17.0
YT
Add the ability to edit playlist items
Add 'Open file' to the context menu
Add the ability to embed thumbnail in the audio/video as cover art
VideoOptionsForm: audio codec does not change when changing audio/video in the video options form

SCrawler
DownloadFeedForm: add ability to merge multiple special feeds into one
AutoDownloader: fix bug when users are added during pool reconfiguration
Scheduler: add the ability to move tasks
FeedMedia: fix image rendering bug
Feed: add select all/none; add the ability to add to a special feed(s) with removal from the current one; add loaded feed name to the title; refresh the loaded feed using the 'Refresh' button
FeedSpecialCollection: add 'Add' button to feed chooser; fixed a bug in the 'Delete' function
SettingsHostCollection, PluginHost: add 'IDisposable' support
API.UserDataBase: add Responser handler options
API.OnlyFans: handle 500 error
API.Threads: extract 'csrftoken' from cookies; simplify 500 error when updating tokens
API.Instagram: update handling of JSON parsing error when downloading reels; fix error downloading single post
API.Facebook: simplify token update errors
API.Twitter: update handling of JSON parsing error
2024-02-18 00:10:39 +03:00
Andy
520280b038 2024.2.12.0
API.Instagram: extract 'csrftoken' from cookies; remove 'x-ig-www-claim' from settings
2024-02-12 10:28:21 +03:00
Andy
10516f229b 2024.2.10.0
Plugins: added `ReplaceInternalPluginAttribute` attribute

SCrawler
API: update user regex for some sites
API.Instagram: simplify 5xx errors; hide JSON deserialization error
API.TikTok: files with long names aren't downloaded (PathTooLongException)
2024-02-10 08:19:08 +03:00
Andy
c3f0831768 2024.2.9.0
Add a 'Feed' button to notifications
SCrawler.StandaloneDownloader: url array form doesn't show scrollbars
2024-02-09 10:00:04 +03:00
Andy
74a0404670 Update README.md 2024-02-03 08:19:53 +03:00
Andy
52a43b9207 2024.1.26.0
YT
YouTubeSettings: add property DefaultVideoFPS
VideoOptionsForm, YouTubeMediaContainerBase: add FPS reduction

SCrawler
API.Instagram: change back aspect ratio determining
API.TikTok: add the ability to use a regex to clean the title
API.YouTube: add the ability to ignore community errors
2024-01-26 04:13:42 +03:00
Andy
5bc559c448 2024.1.20.0
API.Instagram: add reels support (separate)
API.LPSG: handle 404 error
2024-01-20 00:00:23 +03:00
Andy
b37f641582 2024.1.18.0
YT: url array form doesn't show scrollbars
API.Instagram: change aspect ratio determining
API.xHamster: some user videos were not downloaded
API.UserDataBind: incorrect collection sorting
DownloadFeedForm: change separator in result dialog when merging feeds
2024-01-18 01:23:55 +03:00
Andy
e00dfec701 2024.1.12.1
API.Instagram: stories (user) downloading with the wrong aspect ratio for some users
API.YouTube: fix incorrect opening of a post from the feed; fix wrong date to data parsing; add data downloading by dates

DownloadFeedForm: add merging multiple session feeds into one
2024-01-12 19:58:17 +03:00
Andy
94edf23570 2024.1.12.0
DownloadFeedForm: add an option to create a new feed when adding checked items; add a prompt before clearing the current session
MainFrame: add scheduler to tray menu

API.Instagram: fix tagged posts downloading
API.xHamster: fix profiles downloading; add creators downloading
API.YouTube: add error to log (communities)
2024-01-11 23:39:56 +03:00
Andy
0246af9b69 2023.12.27.0
API.OnlyFans: add OF-Scraper support
ProfileSaved: save files when adding new data
2023-12-27 16:10:02 +03:00
Andy
c458f1cd1d 2023.12.26.0
SiteEditorForm: sort controls only if some of them have numbers
UserCreatorForm: reset site options for new user only; add host reset when changing account for created user
PropertyValueHost: add 'UpdateMember' function
SettingsHost: add 'DefaultInstanceChanged' function to update properties when setting default instance
SettingsHostCollection: update properties when setting default instance
2023-12-26 14:34:53 +03:00
Andy
e3da1bf1d3 2023.12.21.0
Update 'new log data' notification
Feed: improve last session loading
2023-12-21 10:01:30 +03:00
Andy
dde024276b 2023.12.20.0
API.Twitter: simplify JSON error string
Add notification of new log data
Add deleting permanent cache path if empty
2023-12-20 15:37:40 +03:00
Andy
7a33c02497 2023.12.15.0
Fix 'GetCurrentMaxVer' bug
2023-12-14 23:51:35 +03:00
Andy
5bd79ff3c2 2023.12.14.1
API.Twitter: add additional nodes
2023-12-14 23:06:55 +03:00
Andy
3268bca0d3 2023.12.14.0
YT
YouTubeSettings: add 'CreateThumbnails_Video' and 'CreateThumbnails_Music' properties
VideoListForm: add 'BTT_SELECT_ALL' and 'BTT_SELECT_NONE'
YouTubeMediaContainerBase: update thumbnails downloading
2023-12-14 14:19:20 +03:00
Andy
1455a31879 2023.12.13.0
YT
Structures: add 'YouTubeChannelTab' enum
YouTubeFunctions: add 'StandardizeURL_Channel' function; update 'Info_GetUrlType', 'Parse' and 'Parse_Internal' functions (YouTubeChannelTab)
YouTubeSettings: add 'ChannelsDownload' property and 'YouTubeChannelTabConverter' converter for grid
Add 'ChannelTabsChooserForm'
VideoListForm: remove buttons 'BTT_ADD_NO_SHORTS' and 'BTT_ADD_SHORTS_ONLY'; update 'BTT_ADD_KeyClick' function

SCrawler
API.M3U8Base: add 'M3U8URL' structure; update 'Download' function with 'OnlyDownload' arg
API.ProfileSaved: add sorting of feed files
API.Xhamster: update M3U8 parser and 'Download' function for additional downloading ways; add 'ReencodeVideos' property (to the 'SiteSettings')
API.YouTube: update to updated 'Parse' function
2023-12-13 19:48:50 +03:00
Andy
64d6e6b28c 2023.12.10.0
YT: move updater functions into the app
SCrawler.API.Twitter: update parsing function to new GDL (1.26.4-dev)
2023-12-10 10:16:58 +03:00
Andy
da7cddc720 Update README.md 2023-12-07 14:08:32 +03:00
Andy
72be6b09ff Update README.md 2023-12-07 11:33:52 +03:00
Andy
3a0837a1b0 2023.12.7.0 2023-12-07 11:03:34 +03:00
Andy
0657f3d195 2023.12.6.1
Update updater
YT: add new version check at start
2023-12-06 18:28:36 +03:00
Andy
c92314d8e8 2023.12.6.0
Add updater
2023-12-06 10:55:50 +03:00
Andy
d99243ce46 2023.12.5.0
YT
VideoListForm: add a check of adding a URL if it has already been downloaded ('ValidateContainerURL')
YouTubeMediaContainerBase: add 'GetUrls' and 'GetFiles' functions; make 'Files' protected friend; update 'CreateUrlFile' function

SCrawler
Add downloaded saved posts to the feed
API.ProfileSaved: add token verification for multi-acc
API.SiteSettingsBase: update 'UpdateResponserFile' and 'CLONE_PROPERTIES.filterUC' functions
API.UserDataBase: add a null host check before request a new key; update 'OpenFolder' function (for saved posts)
API.YouTube: add the ability to download YouTube user community feeds
DownloadProgress: add 'KeyClickEventArgs' to download saved posts excluding from feed; add 'FeedFilesChanged' event; update 'Start' function
DownloadSavedPostsForm: add 'FeedFilesChanged' event and handler; update 'Start' function
Feed.FeedMedia: make the class compatible to work with saved posts
StandaloneDownloader.VideoDownloaderForm: add a check of adding a URL if it has already been downloaded ('ValidateContainerURL')
TDownloader: add the 'IsSavedPosts' field to the 'UserMediaD' structure; update 'UserMediaD.New(EContainer)' function (for saved posts); update 'UserMediaD.ToEContainer' function; add 'SessionSavedPosts' property
MainFrame: add 'Alt+A' hotkey to show scheduler; add 'Alt+P' hotkey to show progress form
Hosts.DownloadableMediaHost: add URL file to files list
2023-12-05 12:05:20 +03:00
Andy
3dae40b696 Update Plugins.md 2023-12-04 07:30:21 +03:00
Andy
ee0c773c37 Update README.md 2023-11-26 03:18:04 +03:00
Andy
ebe5f0ca01 2023.11.25.0
SettingsHost, SiteSettingsBase: change comparer to 'MembersDistinctComparerExtended'
API.RedGifs: fix initial UserAgent value
AutoDownloader: add 'IsManual' to 'Copy' function
2023-11-25 03:00:14 +03:00
Andy
a540aded68 2023.11.24.0
Scheduler: handle scheduler change error (collection was modified)
SchedulerEditorForm: add scheduler name to form title
DownloadProgress: fix disposing error when some objects are already null
API.Reddit: add special notification for error 429
API.Twitter: handle JSON deserialization error
Porn sites: fix incorrect parsing of search queries
YouTube: path not set when adding array to download
2023-11-24 04:54:04 +03:00
Andy
0ec617c1dc 2023.11.21.0
YouTube
Add absolute paths support
PlaylistArrayForm: fix RTB issue
Single media: the file name is not changed manually

SCrawler
Automation: add manual tasks
DownloaderUrlsArrForm: fix RTB issue
SiteSettingsBase: add use of Netscape cookies if enabled for a class; disable saving Netscape cookies on init
Add feeds update when users' location and/or basic info changes
API.TikTok: add ID, username and friendly name extraction from data; update request URL; update 'GetUserUrl' function; add new option 'Use video date as file date'
API.YouTube: set 'UseNetscapeCookies'
2023-11-21 09:25:22 +03:00
Andy
45adf735a7 2023.11.17.0 2023-11-17 02:30:23 +03:00
Andy
496c9487cd 2023.11.15.0
ADD FACEBOOK
SiteSettingsBase: update 'CLONE_PROPERTIES' function (exclude 'DoNotUse' attribute)
API.Instagram: handle 401 error
API.ThreadsNet.SiteSettings: make the class compatible for Facebook
xHanster, XVideos, PornHub, ThisVid: update download function for search queries
Hosts.PropertyValueHost: set the 'Exists' value based on the 'DoNotUse' attribute
Hosts.SettingsHost: use 'GetObjectMembers' instead of 'GetTypeInfo.DeclaredMembers' to get class members
2023-11-17 02:30:23 +03:00
Andy
96705f1c59 2023.11.13.1
PornHub: add playlists downloading
2023-11-13 23:18:21 +03:00
Andy
f08a5f9259 2023.11.13.0
API.JFF: update UserAgent start value
ActiveDownloadingProgress: fix unnecessary focus of inactive form
FeedMedia: add additional buttons to the context menu
FeedSpecialCollection: 'Favorite' feed does not appear in the feeds list if it is created manually
UserCreatorForm: disable account selection when opening an empty form
MainFrame: fix background image resizing on form state changes
SettingsHost: inherit path when cloning an instance
SettingsHostCollection: Extract user path change function to static
2023-11-13 20:53:32 +03:00
Andy
95cbb6aeb1 2023.11.12.2
API.Instagram: handle JSON primitive error (simple line)
API.OnlyFans: handle 401 error; handle file update error
API.Xhamster: handle 503 error
Scheduler: handle automation start error
2023-11-12 22:34:47 +03:00
Andy
5af0dcc46e 2023.11.12.1
YouTube: add cc and subtitles merge
2023-11-12 20:57:05 +03:00
Andy
f5789862ba 2023.11.12.0
Add special feeds
2023-11-12 20:34:53 +03:00
Andy
12c02580f6 2023.11.9.0
ADD MULTI-ACCOUNT

PluginProvider
IDownloadableMedia: added 'AccountName' property
IPluginContentProvider: added 'AccountName' property
ISiteSettings: added properties: 'AccountName', 'Temporary', 'AvailableText', 'DefaultInstance'; added functions: 'Clone', 'Update', 'Delete'; removed 'Load' function; implement 'IDisposable' interface
PropertyValue: added functions: 'BeginInit', 'EndInit', 'Clone'

YT
YouTubeSettings: make the class compatible for multi-acc
YouTubeMediaContainerBase: add 'AccountName' property

SCrawler
IUserData: add properties: 'HostStatic', 'AccountName'
ProfileSaved: add the ability to download saved posts from all accounts
SiteSettingsBase: add multi-acc support; add 'UserOptionsType' for future purposes; update initializers; update responser initializing; add 'CLONE_PROPERTIES' and 'CloneGetEmptySettingsInstance' functions; 'IDisposable' support
UserDataBase: add multi-acc support; change host retrieval method
DomainsContainer: implements 'IDisposable' interface

API.All sites: add multi-acc support; move the Icon and Image setting to the initializer; update initializer
API.Instagram: change some property types
API.Reddit: set 'AvailableText'; update 'UpdateRedGifsToken' and 'UserOptions' functions
API.Mastodon: remove 'MastodonDomains' class and 'SettingsForm' form; replace 'MastodonDomains' with 'DomainsContainer'; update functions 'IsMyUser' and 'IsMyImageVideo'; update to 'DefaultInstance' environment; update 'UserData.ResetCredentials' function
API.XVIDEOS: update to 'DefaultInstance' environment
API.Xhamster: update to 'DefaultInstance' environment

STDownloader: add multi-acc compatibility

SiteEditorForm: add option 'Download saved posts'; update providers; add additional providers; add multi-acc support

PluginsEnvironment: add 'PClonableAttribute'; add multi-acc support
2023-11-12 01:31:09 +03:00
Andy
6def34d5e9 2023.10.12.0
API.ThisVid: update parser ('SessionPosts')
2023-10-12 19:47:02 +03:00
Andy
831d9a5a09 2023.10.10.0 2023-10-10 18:10:45 +03:00
Andy
326e61a968 2023.10.9.0
YT.VideoListForm: hide clear and delete buttons in menu; add 'BTT_CLEAR_SELECTED' button
API.Base.TokenBatch: add debug option
API.ALL: fix missing posts
API.JFF: rewrite m3u8 downloader; add ffmpeg requirement for the download; fixed missing posts; fixed download to the date; fixed corrupted files
DownloadableMediaHost: remove thumbnail when removed from list if thumbnail is stored in cache
2023-10-09 18:52:37 +03:00
Andy
61903afe3f 2023.10.6.0
Collections: use the collection name as a friendly name; labels are removed when creating a new collection
API.UserDataBase: update comparator
API.ThisVid: add 'PagePosts' to continue parsing when new posts are added
2023-10-06 15:11:09 +03:00
Andy
fa3f39b905 2023.10.4.0
YT: file name is missing when destination is changed by selecting one of the saved locations; missing files still appear in the list
API.UserDataBase: hide 'OperationCanceledException' and 'ObjectDisposedException'
API.Reddit: unable to save settings without OAuth data
TDownloader: 'ReconfPool'
MainFrame: replace 'DownloadQueue.FormShow' with 'ShowDownloadQueueForm'
SettingsHost: add a notification if the user has disabled downloading from the site
2023-10-04 22:53:17 +03:00
Andy
c76fd7f918 2023.10.3.0
YT: add settings 'CreateUrlFiles' and 'CreateDescriptionFiles'
STDownloader: add setting 'CreateUrlFiles'
2023-10-03 23:33:42 +03:00
Andy
66085e0d95 2023.10.2.0
API.Threads: save new token if found
Feed: add 'Load current session'; update the session file retrieval function
TDownloader: change session files name; files count
SettingsCLS, GlobalSettingsForm: add session file count setting
2023-10-02 11:32:37 +03:00
Andy
0a1602b453 2023.10.1.0
JustForFans: some profiles won't download
2023-10-01 18:35:08 +03:00
Andy
adf788781d 2023.9.30.1
YouTube: add URL standardization; change the 'Browse' button handler in 'VideoOptionsForm'
API.UserDataBase: add 'TokenQueue' to main function exceptions
2023-10-01 18:02:53 +03:00
Andy
77711965c0 2023.9.30.0
Add Threads.net

API.UserDataBase: add 'EraseData_AdditionalDataFiles' function; add 'ThrowAnyImpl' function; add 'e' arg to 'LogError' function
API.Instagram: make classes compatible with threads.net; add top limits; update container parsers; add an override to the 'EraseData_AdditionalDataFiles' function
2023-09-30 09:16:57 +03:00
Andy
7f1ac6f512 2023.9.29.0
UserDataBind: fix labels, colors, script when adding a new user
UserCreatorForm: fix labels issue
2023-09-29 16:24:43 +03:00
Andy
f4eb33d8da 2023.9.28.0
API.Mastodon: hide 503 error
API.PornHub: minor fixes
API.RedGifs: fix 'DataGone'
Minor bugs
2023-09-28 17:38:34 +03:00
Andy
77443cedc4 2023.9.21.0
PornHub: videos are not downloading
2023-09-21 06:22:17 +03:00
Andy
a446df1f66 2023.9.20.0 2023-09-20 12:17:00 +03:00
Andy
0026e905a4 2023.9.19.0
YT: add priority download protocol
2023-09-19 12:55:53 +03:00
Andy
f8116fd048 2023.9.18.0
API.Instagram: handle error 500; fix saved posts bug
API.Reddit: disable token refresh if there are no profiles to download
API.UserDataBind: consolidate colors; update labels only for the added user
AutoDownloader: change pause event; update pause function for scheduler; fix incorrect pause in scheduler; add icon for SchedulerEditorForm
2023-09-18 07:54:26 +03:00
Andy
8d33fdc8f3 Update README.md 2023-09-18 07:34:00 +03:00
Andy
dab94acc32 2023.9.6.0
Add scheduler changer
2023-09-06 23:38:16 +03:00
Andy
c61c817585 2023.9.3.0
API.Instagram: add user (non-pinned) stories
API.UserDataBase: fix 'StartMD5Checked' initial value
2023-09-03 19:54:01 +03:00
Andy
3ea59a6acd 2023.8.27.0
YT: fix 'Shorts' downloading
2023-09-03 19:38:18 +03:00
Andy
2a60ace18f 2023.8.27.0
API.JFF: remove PXML attribute for some properties
API.Reddit.Channels: save channel info right after download; replace date providers with default
API.Reddit.SiteSettings: improve 'UpdateToken' function
AutoDownloader: add 'Copy' function
SchedulerEditorForm: add cloning plans
DownloadedInfoForm: add 'Try...Catch' for some functions
DownloadFeedForm: add button to go to custom page
FeedMedia: color typo
GroupParameters: add 'ICopier'
2023-08-27 19:39:50 +03:00
Andy
f0014d2874 2023.08.19.0
API.Base: update 'TokenBatch', 'GDLBatch'; add 'YTDLPBatch'
API.Base.UserDataBase: update 'EraseData' function
API.Redgifs: fix hd/sd issue
API.TikTok: update dates in commands; replace 'TokenBatch' with 'YTDLPBatch'
UserDownloadQueueForm: update functions
2023-08-19 18:42:01 +03:00
Andy
28ae44f0ae 2023.08.17.0
YT.VideoListForm: hide progress

API.Base.UserDataBase: add 'IsUser' property; remove 'DownloadedPictures' debug line; add a special log for non-existent users
API.Twitter: group 'limit' notifications; update 'TwitterLimitException' (inherits Plugin.ExitException)
AutoDownloader: fix 'Initialization' value bug
DownloadedInfoForm: fix a bug due to which profiles were disposed
FeedMedia: add subscriptions users BackColor & ForeColor; fix file name issue; remove icon cloning
TDownloader, UserDownloadQueueForm: fix progress hang issue
ColorPicker: add 'TooltipText'
GlobalSettingsForm: add new properties; move design properties to new tab
ListImagesLoader: add subscriptions users BackColor & ForeColor
2023-08-17 23:54:19 +03:00
Andy
1b1226025a 2023.08.10.0
Add JFF

Update groups
Add advanced filter
Add advanced download
Disable 'ShowInTaskbar' on several forms
API.Base.M3U8: add external cache support
API.Base.UserDataBase: update token names
Feed.FeedMedia: add clone icon
UserCreatorForm: fix bug collection labels not showing
MainFrame: update 'DownloadSiteFull' function
UserSearchForm: move focus to textbox on form is open
2023-08-10 22:42:29 +03:00
Andy
58927b3113 Update README.md 2023-08-08 17:17:23 +03:00
Andy
df06a86651 2023.8.6.0
Plugins.Attributes: add 'DependentFields' attribute
Plugins.IPluginContentProvider: add 'Options' and 'IsSubscription' properties
Plugins.ISiteSettings: add 'SubscriptionsAllowed' property
Plugins.ExchangeOptions: add 'Options' field
Plugins.Attributes.PropertyUpdater: replace 'Dependencies' with 'Arguments'

YT: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties; add the ability to store download locations; add 'DownloadLocation' and 'DownloadLocationsCollection' objects
YT.IDownloaderSettings: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties
YT.Downloader: fixed bug with re-saving elements when loading a video list; fixed bug when files were not deleted when clicking on the delete button; fixed a bug that caused the video to redownload; download job removes elements at wrong indexes; added skipping of downloaded elements in the job; fixed a bug, pending option did not change after download complete
YT.YouTubeMediaContainerBase: add '_MediaStateOnLoad' field and 'NeedToSave' function; update the 'Save' function to prevent saving a file when a download is complete and the file has already been saved; update code for new yt-dlp version

Fixed cache deletion errors
Add user queue
Add global locations
API.Base.SiteSettingsBase: implement 'SubscriptionsAllowed' property; remove request headers with null values on save; add '_AllowUserAgentUpdate' parameter
API.Base.Structures: add 'SiteModes' enum
API.Base.UserDataBase: add 'Erase' button; implement 'Options' and 'IsSubscription' properties; add 'SpecialLabels' property; update 'LVIKey'; update 'FitToAddParams' function; add 'EraseData' function; user colors; Not UserExists notification, UserQueue support
API.Base: add 'DeclaredNames'
API.Instagram: remove default values for headers; disable updating UserAgent from global; check for a new username for non-existent users
API.Mastodon: bypass new inherited twitter options; update names and headers
API.OnlyFans: make 'HH_BROWSER' property nullable; remove 'HH_BROWSER' from required; fix username bug (dots); handling of 504 and 429 errors; add 'DownloadHighlights' and 'DownloadChatMedia' options; add 'UserExchangeOptions'; fixed incorrect error handler
API.PathPlugin: fixed incorrect detection of path existence
API.Pinterest: add 'SpecialLabels'
API.PornHub: add new video regex; remove old regex; added 'DownloadUploaded', 'DownloadTagged', 'DownloadPrivate' and 'DownloadFavorite' properties to 'SiteSettings', 'UserData' and 'UserExchangeOptions'; update regex to define user; added downloading search queries; update 'GetUserUrl' function; hide unnecessary 'RegexFieldsTextBecameNullException' errors; add subscriptions
API.Reddit: add 'SpecialLabels'; add bearer token and its refresh interval; add OAuth; add additional options
API.RedGifs: add 'DependentFields' for 'Token'
API.ThisVid: add 'DownloadFavourite' option; add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; updating cookies issue
API.TikTok: rewrite algorithms
API.Twitter: add 'UseAppropriateModel', 'UseNewEndPointSearch', 'UseNewEndPointProfiles', 'AbortOnLimit', 'DownloadAlreadyParsed', 'MediaModelAllowNonUserTweets' properties; remove old commented code; remove 'TwitterPic_400' and replace with 'TwitterIcon_32.ToBitmap'; add 'DownloadModelForceApply' user option; update environment to GDL 1.25.8; fixed gifs downloading; fix typo in 'ReparseMissing'; update names
API.UserDataBind: prevent adding site-specific labels when adding to a collection
API.Xhamster: add downloading search queries, tags, categories; add 'SpecialLabels'; add additional nodes for channels; add subscriptions
API.XVIDEOS: add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; changed users creation method; add subscriptions
API.YouTube: add subscriptions
AutoDownloader: add new group subscription options; update predicates; fixed excluded labels and sites in default mode; update notifications; add an additional skip options, add 'Force start' option
DownloadedInfoForm: add subscriptions; fixed size/location bug; hide unnecessary error (refill)
Feed: add subscriptions; update filters; add 'Ctrl+G' shortcut
FeedMedia: add subscriptions; fixed 'webm' bug; add title for subscription media; add site icon to post; user colors; always using 'FriendlyName' instead of 'UserName' if it exists
DownloadGroup, GroupDefaults, GroupParameters: add subscription and 'UsersCount' options
MissingPostsForm: add 'BTT_DELETE_ALL'
VideoDownloaderForm, DownloaderUrlForm, DownloaderUrlsArrForm: add download locations support
VideoDownloaderForm: add subscriptions support
GlobalSettingsForm: add new properties
UserCreatorForm: add subscriptions; add 'Options' support (of 'ExchangeOptions'); user colors
ListImagesLoader: add subscription colors; user colors
MainFrame: add subscriptions; add filters by subscription and user; update predicates
NuGet: update 'LibVLCSharp', 'LibVLCSharp.WinForms', 'VideoLAN.LibVLC.Windows'
DownloadableMediaHost: update 'Save' function
PropertyValueHost: fix 'CaptionWidth' bug; add 'Dependents'
SettingsHost: add 'Dependents'
UserDataHost: add 'Options' and 'IsSubscription' properties
SettingsCLS: implement new 'IDownloaderSettings' properties; add 'CacheSnapshots'; add 'DownloadLocations'; add new properties
UserInfo, UserFinder: add subscriptions
UserSearchForm: fixed search by name bug
2023-08-06 18:16:07 +03:00
Andy
bade8666d5 Update README.md 2023-06-29 20:54:27 +03:00
Andy
c70caa0035 Update README.md 2023-06-23 09:40:40 +03:00
Andy
ac532dbc6f Update README.md 2023-06-21 14:00:25 +03:00
Andy
82ef4f4410 2023.6.19.0
YT.Progress: make the playlists parsing progress more informative; change form display method
YT.YouTubeMediaContainerBase: fix sort algo
YT.Tray: add 'Add' button; add 'Ctrl+Click' on tray icon to add download
YT.Settings: add setting 'Download on click in tray: show form'
LPSG: some files didn't download (encoding)
Twitter: hide cache deletion errors
Mastogon: fixed bug in 'ReparseMissing' function
Reddit: downloaded gifs are static
XHamster: videos are not downloading or downloading incorrectly
Progress: fix bugs; minor improvements
2023-06-19 06:05:28 +03:00
Andy
d34414359c 2023.6.9.0
YT.MediaItem: fixed opening paths to downloaded playlists and channels
API.InternalSettingsForm: add members distinct
API.Mastodon: create personal EditorExchangeOptions class with some Twitter members disabled
API.Twitter: add 'DownloadModels'; update algo
Make progress more informative
2023-06-09 21:44:00 +03:00
Andy
e51debc027 2023.6.8.0
YT.Music: append artist name to music playlist output path
YT.MediaItem: fixed opening paths to downloaded playlists and channels
YT.YouTubeMediaContainerBase: save thumbnail path for playlist and channel
UserDataBase: remove old line of code
API.Twitter: fixed profile not fully downloaded
SiteEditorForm: corrected form size for small monitors
2023-06-08 17:27:19 +03:00
Andy
938042ea9e 2023.6.5.0
YT settings: removed property 'ItemsListLimit', add property 'ReplaceModificationDate'
YT.MediaItem: fix 'Pending'
YT.VideoListForm: add 'Shift' to add without downloading; add 'F5' hot key to start download; remove list items limit; fix item 'Pending', fixed items queue

UserDataBase: add 'IconBannerDownloaded' properties; add 'HOST.Available' check to 'DownloadSingleObject'; update file deletion in 'DownloadContentDefault'; add truncating '_TempPostsList' if number of ids > 1000
Instagram: add authorization headers
Mastodon: implement 'DownloadIconBanner'; update 'ReparseMissing' function
Reddit: implement 'DownloadIconBanner'
Twitter: implement 'DownloadIconBanner'; update parsers to parse posts with two videos; implement gallery-dl for all function; remove headers from settings
Download.DownloadProgress: remove main progress perform when downloading saved posts
VideoDownloaderForm: bind the 'BTT_ADD_URLS_ARR' button to the 'BTT_ADD_KeyClick' function
UsersInfoForm: add folder opening on double click on an item
ListImagesLoader: fix refill bug when the number of filtered profiles = 0
TrayIcon: add standalone downloader to context menu
DownloadableMediaHost: fix a bug when not downloaded videos do not appear in the list when loading the program
2023-06-05 19:36:35 +03:00
Andy
abdef81e5f Update README.md 2023-06-03 11:51:01 +03:00
Andy
e868c2e694 2023.5.12.0
IPluginContentProvider: add 'ProgressPreChanged' and 'ProgressPreMaximumChanged' events
YT.MediaItem: change folder opening on double click
YT.VideoListForm: change the icon for the 'Download' button

Add advanced progress
Add user metrics calculation
UserDataBase: fix GIF hash bug
Instagram: heic to jpg
Mastodon.SiteSettings: add the main domain to the list of domains with saving the settings
Mastodon.UserData: handle 'Forbidden' error; fix bug in parsing non-user posts
Pinterest: remove cookies requirement for saved posts
PornHub: fix resolutions issue; add 'DownloadUHD' option
Reddit: fix missing images bug; fix broken images bug; update container parsing function
MainFrame: fix collection pointing bug
2023-05-12 20:00:32 +03:00
Andy
b2a9b22478 2023.4.28.0
Plugins
IPluginContentProvider: added DownloadSingleObject function; added tokens to GetMedia and Download functions; removed GetSpecialData function
Add IDownloadableMedia interface
Removed 'Channel' option from all functions and enums
ISiteSettings: added GetSingleMediaInstance function
ExchangeOptions: removed 'IsChannel'
UserMediaTypes: added Audio and AudioPre enums
IUserMedia, PluginUserMedia: changed ContentType and DownloadState from integers to their enums

SCrawler
Add YouTube standalone downloader
Add gallery-dl & yt-dlp support
Remove 'UserInfo' requirement from 'ProfilesSaved'
Update 'SiteSettingsBase' to use domains and Netscape cookies
UserDataBase: remove channels; remove old 'Merge' const; standardize SavedPosts file naming; move 'ValidateMD5' function from Twitter to UserDataBase to use it in other UserData classes; add 'DownloadSingleObject' environment for single posts; add validating file extension for m3u8 during download; add reindex of video file during download

Rewritten DomainsContainer
Create a universal settings form and PSettingsArttribute
Gfycat, Imgur: turn these classes into IUserData to download a single object

All plugins: update 'GetInstance' function for saved posts; update domains where implemented; remove 'OptionForm' where it exists; update options where they exist; update unix date providers; reconfigure channels where they exist

LPSG: fix attachments; update converters and regex
Add sites: ThisVid, Mastodon, Pinterest, YouTube, YouTube music
Reddit: standardize container parsing for all data types; new channel environment; fix 'ReparseMissing' function; redirect data downloading to the base download function, saved crossposts support
Twitter: fixed gif path bug; fixed downloading saved posts
PornHub: hide unnecessary errors; photo galleries bug
RedGifs: add 'UserAgent' option

Added icons to download progress

Rename some objects
Completely redesigned standalone downloader form and rewritten its environment
WebClient2: update to use tokens

Labels: update label form (save labels to file only when OK button is clicked); change removing labels.txt from recycle bin to permanent; disable storing label 'NoParsedUser'

UserCreatorForm: remove the 'Channel' checkbox and related functions; ability to extract the user's URL from the buffer and apply parameters if found
Remove temporary 'EncryptCookies' module

MainFrame: added simplified way to create new users (Ctrl+Insert to create a new user with default parameters from clipboard URL); removed SCrawler command line argument "-v" (remove the ability to run SCrawler as video downloader)
PropertyValueHost: update for option forms compatibility
SettingsHost: removed 'GetSpecialData' fork; added 'GetSingleMediaInstance' fork
UserDataHost: update functions with tokens; update events; add 'DownloadSingleObject' function
Settings: add the ability to get environment from 4 destinations; add the ability to set the program environment manually; add CMDEncoding; add cache; remove the old function 'RemoveUnusedPlugins'; add 'STDownloader' properties; add YT compatibility; add new notification options; add deleting user settings file when 'SettingsCLS.Dispose()' if where are no users in SCrawler
UserFinder: remove old 'Merge' const; remove channel option
UserInfo: remove channel option
2023-04-28 10:13:46 +03:00
Andy
db9e2cfb88 2023.3.5.0
Fixed SavedPosts new file naming method
Fixed Twitter MD5 comparison error
Fixed ffmpeg file parts concatenation algorithm
2023-03-05 06:12:08 +03:00
Andy
85d8df96ca 2023.3.1.0
Add 'Path' plugin
UserDataBase: changed file names for saved posts; removed 'Self' property; add 'MyFileSettings' field; added UserSiteName; changed download envir algo
Twitter: added MD5 comparison; duplicate images removal option; UserSiteName parsing; download icon and banner
Instagram: added a new option for token 'www_claim'; removed requirement of token 'www_claim'; UserSiteName parsing; download icon
Reddit: UserSiteName parsing; download icon and banner
PornHub: fixed unicode titles
XHamster: added channels
ffmpeg: fixed max input length error during files combining; fixed encoding issue
Feed: added images centering; added BackColor and ForeColor change
MainFrame: added BackColor, ForeColor, and BackgroungImage change; added 'UpdateLogButton' when load completed
ListImagesLoader: fixed wrong notification when no users found
SettingsCLS: updated users loading algo
2023-03-01 20:35:52 +03:00
Andy
6ca90f0489 Update names 2023-02-25 10:19:18 +03:00
Andy
2a780a3acf Update README.md 2023-02-13 11:44:00 +03:00
Andy
290b5c4586 2023.2.5.0
Update to library environment
GlobalSettings: add UserAgent
Gfycat: fix RedGifs urls issue
2023-02-05 20:58:02 +03:00
Andy
f5e68a7032 Update Changelog.md 2023-01-28 05:50:07 +03:00
Andy
ece573dd40 Update README.md 2023-01-28 05:48:54 +03:00
Andy
1f1148020c 2023.1.27.0
Plugins: added 'Interaction' to 'Provider' attribute; added 'IPropertyProvider' interface
Hosts: update classes to work with new options
Instagram: fixed pinned post reload
Twitter: advanced options for GIFs
UserCreatorForm: change icon based on the selected site
UserSearchForm: change search function
2023-01-27 16:43:57 +03:00
Andy
fc226d549a 2023.1.24.1
Some Imgur albums won't download
Added icon for standalone downloader
2023-01-24 16:13:46 +03:00
Andy
602771d982 2023.1.24.0
Imgur albums not downloading
Collections: users in the collection are not banned
2023-01-24 06:05:40 +03:00
Andy
3e472b4f2b Update HowToSupport.md 2023-01-13 00:21:48 +03:00
Andy
30c3fe3b68 Update info
Update info
2023-01-12 07:38:17 +03:00
Andy
38c81b7a0b 2022.1.2.0
Redgifs: added token refresh interval; reduced interval value
Updated labels collection
PornHub: fixed bugs
Notifications: pressing any button opens SCrawler
User list loader finished
2023-01-02 18:53:24 +03:00
Andy
0fb6add751 Update UserData.vb 2022-12-27 15:19:40 +03:00
Andy
5d64b8c7ce 2022.12.27.0
XVideos: added 'Quickies'; fixed downloading.
Instagram: added more enable/disable options.
2022-12-27 15:04:56 +03:00
Andy
aabf6d62ab 2022.12.26.0
UserMedia: fixed plugin bugs
Instagram: updated algo and settings; update responser settings based on site response
PornHub: fixed bug in SiteSettings; fixed typos
RedGis: fixed downloading user profiles
XVideos: fixed user profile opening
UserDataBind: fixed multiple collection removing issue
DownloadedInfoForm: fixed user focusing
UserCreatorForm: add user name to form header if user exists
ListImagesLoader: changed loading algo
MainFrame: added channels button to tray context menu
Added ffmpeg fox x86
Fixed typos
2022-12-26 17:37:25 +03:00
Andy
03487185c5 Update names
Updated library objects
2022-12-24 15:45:12 +03:00
Andy
f0686bbc8e Fixes
Fixed typo
Added label icon to context menu
Fixed PornHub bug
Added 'Object' to IUserMedia
2022-12-24 15:18:04 +03:00
Andy
bdc7321331 2022.11.16.0
Add sites: PornHub, XHamster
Add saved xvideos posts downloading
PluginProvider: added TaskGroup attribute; added IUserMedia inteface; changed PluginUserMedia to IUserMedia in interface declarations; changed 'User' String to IPluginContentProvider in ISiteSettings sinterface
Added update the 'LOG' button at the end of the ProfileSaved download function
API.Base: added 'IUserMedia' compatibility for 'UserMedia'; moved 'GetImage' from 'UserPost' to 'ChannelsViewForm'; update constants in UserDataBase; updated UserDataBase to new UserInfo environment.
API.Instagram.UserData: fixed date issue
API.Reddit.SiteSettings: update user patterns
API.Twitter.Declarations: moved provider here from MainFrame
UserDataBind: updated to new UserInfo environment
ActiveDownloadingProgress: updated form rendering
AutoDownloader: added SpecialDelay
TDownloader: added 'Suspended' option; updated for TaskGroups
CollectionEditorForm: fixed order bug
LabelsForm: remove old stuff
UserEditorForm: added collection editing
MainFrame: improve label selection
Add import users
Added the ability to create a virtual collection and add a virtual user to a real collection
SettingsCLS: improve users loading
2022-11-16 13:41:45 +03:00
Andy
7d169acebc 2022.10.23.0
PluginProvider: added 'DoNotUse' attribute.
Channels: copying a channel to the 'ChannelsDeleted' folder before deleting.
Twitter: updated status codes.
AutoDownloader: removed base parameters initialization; updated 'ToEContainer' function; 'StartupDelay' default value = 1; added IIndexable; fixed NextExecutionDate; added task delay based on index and tasks count; updated user selections algorithms.
MainFrame: coloring the button 'Download All' depending on the pause; updated user selections algorithms.
DownloadGroups: added LabelsExcluded, Sites and SitesExcluded; updated initialization; updated ToEContainer function; updated user selections algorithms
IGroup, GroupParameters: added LabelsExcluded, Sites and SitesExcluded; added Import and Export functions
Removed TrayIcon notifications. All notifications are now ToastNotifications.
SettingsHost: 'DoNotUse' attribute
Settings: added 'GetUsers' predicate function
2022-10-23 16:39:01 +03:00
Andy
f5c156b8e5 2022.10.18.0
Moved UserMedia xml initialization to the structure itself
Added download with feed skip
Added silent mode (temporary disabling notifications)
Added additional Instagram protection
Excluding users whose profiles do not exist from downloading with groups and AutoDownloader
Feed: delete file bugs; reorder data after file deletion; video playback bugs
SiteSettingsForm: enable 'OK' button when editing cookies
Fixed collection users ban
Settings: disabling ffmpeg missing notification; advanced notification management
Added 'ToolStripKeyMenuItem' control
Plugins: deprecated XVIDEOS and LPSG plugin libraries; moved them to SCrawler.
Updated license
PluginProvider: added 'BeginEdit' and 'EndEdit' function to ISiteSettings; changed GetSpecialData (ISiteSettings) return type to IEnumerable
PluginsEnvironment: removed 'IsMyClass' attribute
MainFrame: grouped all download buttons into one menu; reorganized code; removed 'F2' hotkey
AutoDownloader: added advanced pause options; added buttons to tray icon and AutoDownloader form
MissingPosts: finished; activated functions that were disabled; added download functions to UserData classes
UserDataBase: ability to use responser; ability to download m3u8; extended 'DownloadingException' with optional argument 'EObj'; user index in collection (button tag) changed to user instance; extended information with user labels; updated 'ProcessException' function
Replaced download buttons with 'KeyClick' control
Replaced FDatePickerForm with my library's form
Collections: Deleting multiple collections - disabled confirmation; ban each user in collection
2022-10-18 12:05:31 +03:00
Andy
d91ee72eaa Update LICENSE 2022-10-18 12:02:43 +03:00
Andy
4e9de23b60 Deprecated
Moved to SCrawler
2022-10-18 11:54:50 +03:00
Andy
129558c262 2022.9.24.0
Fixed wrong image opening in Autodownloader
Fixed incorrect feed grid resizing when removing media
Fixed incorrect removal of users from the collection
Fixed Instagram function displaying number of requests: wrong value type.
Fixed XVIDEOS cycle bug
Collection: add multiple users
Collection: new collections at the top
Copying user data
Feed: 'Season' and 'Date' to the post title.
2022-09-24 20:26:40 +03:00
Andy
a3e79eb4bc Update UserDataBase.vb
Fixed typo
2022-09-17 20:28:27 +03:00
Andy
eb28255de3 2022.9.17.0
Extended filters by date
Added download by dates for multiple users
Changed validation of dates ranges in UserDataBase
Add user filters by dates
Add disabling site downloading
Fixed Twitter date validator
2022-09-17 19:59:55 +03:00
Andy
92be0994ae 2022.9.16.0
Removed some compatible functions
Fixed Settings.GetUser bug
Design improvements
Changed UserMediD comparer
FeedVideo design updated, incorrect time position fixed, bugs fixed
Fixed getting Reddit channel video thumbnail
2022-09-16 19:41:24 +03:00
Andy
9567b0a367 2022.9.13.0
Added video duration to the feed
Added skipping of pinned Instagram posts if they are already downloaded
2022-09-13 16:20:07 +03:00
Andy
c28c0e1ba3 2022.9.10.0
Fixed: missed posts are not saved
Fixed memory leaking because of the video
2022-09-10 12:28:40 +03:00
Andy
86771eee94 2022.9.8.1
Fixed unexpected memory leak when using the 'Feed' form
2022-09-08 22:24:36 +03:00
Andy
02e8a15ae3 2022.9.8.0
Temporary disabled RedGifs downloading
Added 'missing posts', 'feed'
Fixed minor bugs
2022-09-08 12:36:25 +03:00
Andy
443ab329d5 2022.8.28.0
Changed target platforms
Added RedGifs pics
Fixed Switcher limit bug
2022-08-28 04:08:54 +03:00
Andy
a16bb8de90 Update CONTRIBUTING.md 2022-08-26 20:38:33 +03:00
Andy
0af5e6f8d4 Update README.md 2022-08-26 20:37:54 +03:00
Andy
54ffe10f71 2022.8.22.0
Cleaned up the code
Replace some old functions with new ones
Adapted to the new library environment
Enable/Disable display user/downloaded image
Autodownloader option 'Show notification' not saved
Separate thread for standalone video downloader
Expanded the description of some errors with additional information
Fixed date/time renaming issue
Fixed internal library bugs
Fixed minor bugs
2022-08-22 02:42:36 +03:00
520 changed files with 62861 additions and 16867 deletions

3
.gitattributes vendored
View File

@@ -1,2 +1,5 @@
# Auto detect text files and perform LF normalization
* text=auto
# Declare files that will always have CRLF line endings on checkout.
*.md text eol=crlf

4
.github/FUNDING.yml vendored
View File

@@ -3,11 +3,11 @@
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: andyprogram
ko_fi: #andyprogram
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
otechie: # Replace with a single Otechie username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
custom: ['https://blockchair.com/bitcoin/address/BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET']
custom: ['https://github.com/AAndyProgram/SCrawler/blob/main/HowToSupport.md']

View File

@@ -13,11 +13,16 @@ A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. **Profile URL**:
2. Do something
3. See error
2. **Post URL**:
3. Do something
4. See error
**Log data**
If the program log contains any data.
<details>
<summary>Log data</summary>
<pre>
If the program log contains any data, replace this line with the log data. If the program log does not contain any data, write here about.
</pre>
</details>
**Expected behavior**
A clear and concise description of what you expected to happen.
@@ -25,11 +30,17 @@ A clear and concise description of what you expected to happen.
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Release information (please complete the following information):**
- OS [e.g. Windows 10, Windows 11]
- Architecture [e.g. x86, x64]
- Version [e.g. 2.0.0.0]
- NET.Framework version
**Release information:**
**Please complete the following information or replace the following text with data copied from SCrawler (click the top right info button in the main window, then the `Environment` button, then the `Copy` button, and paste the copied text here).**
- OS: [e.g. Windows 10, Windows 11]
- Architecture: [e.g. x86, x64]
- Version: [e.g. 2023.3.5.0]
- NET.Framework version:
- ffmpeg version (command `ffmpeg -version`):
- yt-dlp version (command `yt-dlp --version`):
- gallery-dl version (command `gallery-dl --version`):
- cURL version (command `curl --version`):
**Additional context**
Add any other context about the problem here.

5
.gitignore vendored
View File

@@ -9,8 +9,8 @@
*.user
*.userosscache
*.sln.docstates
ffmpeg.exe
ToDo.txt
.obsidian/
BugReporterFormDiscordWebHook.vb
# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs
@@ -33,7 +33,6 @@ bld/
[Oo]bj/
[Ll]og/
[Ll]ogs/
ffmpeg/
Info/
# Visual Studio 2015/2017 cache/options directory

View File

@@ -5,19 +5,30 @@ I welcome requests! Follow these steps to contribute:
1. Find an [issue](https://github.com/AAndyProgram/SCrawler/issues) that needs assistance.
1. Let me know you are working on it by posting a comment on the issue.
1. If you find an error in the code, please provide a link to the file and the line number.
1. If you have a code change suggestion, you can post a replacement code block. I also accept pull requests.
1. If you have a code change suggestion, you can post a replacement code block.<!-- I also accept pull requests.-->
# How to report a problem
1. Attach the **profile URLs or links** that you cannot download.
1. Attach the **LOG** if it exists.
1. **Attach the environment information copied from SCrawler (click the top right info button in the main window, then the `Environment` button, then the `Copy` button, and paste the copied text into the issue).**
1. *Add screenshots to illustrate the problem (**optional**)*
# How to build from source
1. Delete the `PersonalUtilities` project from the solution.
1. Delete the `PersonalUtilities.Notifications` project from the solution.
1. The following libraries must be added to project references with the '**Copy to output folder**' option:
- `PersonalUtilities.dll`
- `PersonalUtilities.Notifications.dll`
- `Microsoft.Toolkit.Uwp.Notifications.dll`
- `System.ValueTuple.dll`
1. Import `PersonalUtilities.Functions` for the whole project.
1. Delete the "PersonalUtilities" project from the solution.
1. Delete the "PersonalUtilities.Notifications" project from the solution.
1. Add the latest versions of the ```PersonalUtilities.dll``` and ```PersonalUtilities.Notifications.dll``` libraries (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
1. Import PersonalUtilities.Functions for the whole project.
**Always use the correct "PersonalUtilities.dll" library. You must download this library from the release of the code you downloaded.**
**Always use the correct libraries. You must download libraries from the same release date as the code commit date.**
# How to request a new site
**I'm currently not accepting requests to develop new sites.**
1. Check [issues](https://github.com/AAndyProgram/SCrawler/issues) (open and [closed](https://github.com/AAndyProgram/SCrawler/issues?q=is%3Aissue+is%3Aclosed)) and [discussions](https://github.com/AAndyProgram/SCrawler/discussions) to find your issue. Perhaps I have already answered your request.
1. If you don't find anything, create a new issue with your request. I usually reply as soon as possible (within the next few hours).
@@ -34,14 +45,4 @@ I welcome requests! Follow these steps to contribute:
If I'm interested in a site you want to add, it may be added in future releases.
# Sites I will never develop
- Facebook
# Sites requested by users
- TikTok
- API for receiving data without authorization was not found. Therefore, I don't have time to start developing this site parsing algorithm. If anyone knows of requests that may collect data without OAuth authentication, please let me know.
# Contact me
[![matrix](https://img.shields.io/badge/Matrix-%40andyprogram%3Amatrix.org-informational)](https://matrix.to/#/@andyprogram:matrix.org)
- Tumblr

File diff suppressed because it is too large Load Diff

54
FAQ.md
View File

@@ -16,12 +16,6 @@ A: https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies
----
#### Q: **I can't copy cookies.**
A: Use the mouse. Don't use ```Ctrl``` + ```A```!
----
#### Q: **Does this program have GUI or CLI.**
A: This is a GUI program.
@@ -30,25 +24,31 @@ A: This is a GUI program.
#### Q: **Will CLI be added in the future?**
A: I do not think so.
A: NO.
----
#### Q: **I want to add "...." site. How to request.**
A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-a-new-site)
<!---A: How to request a new site you can read [here](CONTRIBUTING.md#how-to-request-a-new-site)--->
**I'm currently not accepting requests to develop new sites.**
----
#### Q: **Twitter/Instagram download failed.**
#### Q: **Site download failed.**
A: Check your credentials. Both of these sites require cookies. Check your [Twitter tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) and [Instagram settings](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-settings). If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
A: Check your credentials and **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**. If all settings are set, but nothing works, go to [create a new issue](https://github.com/AAndyProgram/SCrawler/issues). Don't forget to attach the LOG.
**You also can join our Discord server**: https://discord.gg/uFNUXvFFmg
<br/>*You can get help faster there!*
**ATTENTION! Issues without URLs will be closed without a response!**
----
#### Q: **I have set credentials but still nothing is downloading**
A: Click the ```Start downloading``` button
A: Click the `Start downloading` button or press `F5`
----
@@ -60,7 +60,7 @@ A: https://github.com/AAndyProgram/SCrawler/releases/latest
#### Q: **How to run the program?**
A: Double-click ```SCrawler.exe```
A: Double-click `SCrawler.exe`
----
@@ -78,22 +78,44 @@ A: The program stored posts IDs in users' folders. For the first time, the progr
#### Q: **How to redownload all data**
A: Double-click on the user you want to redownload. In the opened window open folder setting. Delete the files ending with ```_Data.xml``` and ```_Posts.txt```. Download this user again.
A: https://github.com/AAndyProgram/SCrawler/wiki#redownload-user
----
#### Q: **How to remove the label**
A: There is no functionality to remove an individual label. You can open the ```Labels.txt``` file in the program settings folder and delete any label you want. You also can delete this file (```Labels.txt```). In this case, when the program starts, the list of labels list will be updated with only existing labels (from the user data files).
A: There is no functionality to remove an individual label. You can open the `Labels.txt` file in the program settings folder and delete any label you want. You also can delete this file (`Labels.txt`). In this case, when the program starts, the list of labels list will be updated with only existing labels (from the user data files).
----
#### Q: **How to remove a user from the blacklist**
A: Just add that user back to the program. In the dialog box that opens, click on the ```Add and remove from blacklist``` button.
A: Just add that user back to the program. In the dialog box that opens, click on the `Add and remove from blacklist` button.
----
#### Q: **Why don't you answer how it works**
A: Because **I don't want to**. I don't want to waste my time explaining things that are already covered in the **[GUIDE](https://github.com/AAndyProgram/SCrawler/wiki)**! If you didn't bother to read the guide, why would I waste my time?! ALL FUNCTIONALITY IS DESCRIBED IN THE GUIDE. Before publishing a new release, I update the guide. If you don't respect my work, I don't waste my time.
----
#### Q: **You lost me. Your program is too complicated.**
A: **I'm fine with that**. If the program is difficult for you or you can't configure it, I can only suggest you find another (easier) program. I really don't mind! The program is free. I am develop SCrawler for myself and publish on GitHub because people found my program useful. If someone can't use it or doesn't like it, I'm fine.
----
#### Q: **I can't configure something**
A: I can only [suggest](#q-you-lost-me-your-program-is-too-complicated) you find another (easier) program.
----
#### Q: **Can you add a step-by-step guide or video on how to use the program?**
A: **NO**! I will not do it. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.
A: **NO!** The guide fully covers all the functionality of SCrawler! If you don't respect my work, I don't waste my time. If you want, you can create a video tutorial and send it to me. Then I add it. All options and what each option does described on the wiki. The wiki also contains a description of all settings and how-to configure them. For complex settings, there is a steep-by-steep guide. Read the [main](README.md) information and [GUIDE](https://github.com/AAndyProgram/SCrawler/wiki/) and you won't have any problems. I have developed a program with an intuitive interface. There is a Settings button, download buttons, a context menu that drops down when a user is clicked, and other controls. Anyone can use it.
**The following video was recorded by a user who loves SCrawler and demonstrates how to add credentials using Instagram as an example:**
[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)

View File

@@ -1,11 +1,12 @@
Your support is very valuable to me. Any support is greatly appreciated. Your support encourages me to make new features, update the program, add new sites, etc.
You can support the program by:
- **Bitcoin**: bitcoin:BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
- :heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram
- **Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
- :repeat: make a post about my program on your profile (Reddit, Twitter, Instagram and any other social networks)
- :speech_balloon: tell your friends about the program
- :heart: like the program on this site: https://alternativeto.net/software/scrawler/about/
- :heart: like the program on this site: https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml
- suggest my program as an alternative ([on this site](https://alternativeto.net/software/scrawler/about/)) to any program you have used before
I would be very grateful for any support! :blush:
<!---:heavy_dollar_sign: make a donation on this site: https://ko-fi.com/andyprogram--->

View File

@@ -1,6 +1,10 @@
You can create a plugin for any site you want. **To create a plugin, read [this guide](https://github.com/AAndyProgram/SCrawler/wiki/Plugins).**
If you've created a plugin, you can create a [new issue](https://github.com/AAndyProgram/SCrawler/issues/new?assignees=&labels=New+Plugin&projects=&template=plugin_add.md&title=%5BNEW+PLUGIN%5D) and I'll add your plugin to the list below.
----
List of available plugins:
- LPSG
- XVIDEOS
Tools:
- [image2post](https://github.com/unknown81311/SCrawler-image2post) by @unknown81311: **get reddit post URL from file.**

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 17 KiB

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 472 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 101 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.4 KiB

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 369 KiB

After

Width:  |  Height:  |  Size: 359 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 379 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.7 KiB

After

Width:  |  Height:  |  Size: 39 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 18 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 10 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 6.3 KiB

After

Width:  |  Height:  |  Size: 7.7 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.6 KiB

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 26 KiB

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 14 KiB

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 22 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 17 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 21 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

View File

@@ -1,6 +1,63 @@
# yt-dlp
https://github.com/yt-dlp/yt-dlp/
**Great powerful CLI tool that supports hundreds of sites.**
SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.
# gallery-dl
https://github.com/mikf/gallery-dl
**Great powerful CLI tool that supports hundreds of sites.**
SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.
# 4K Video Downloader
https://www.4kdownload.com/-plbrz/video-downloader
| Option | SCrawler | 4K Video Downloader |
| ---- | ---- | ---- |
| User managament | **Advanced** | No |
| Automatic downloads | **Yes** | No |
| Downloading groups | **Yes** | No |
| Labeling users | **Yes** | No |
| Filtering | **Yes** | No |
| Collections | **Yes** | No |
| Specific user folders | **Yes** | No |
| Favorite / Temporary user options | **Yes** | No |
| Plugins support | **Yes** | No |
| Download single video | **Unlimited** | 30 videos per day *(unlimited starts from 12 EUR)* |
| Download videos per channel | **Unlimited** | 5 free *(unlimited starts from 12 EUR)* |
| Download videos per playlist | **Unlimited** | 10 free *(unlimited starts from 12 EUR)* |
| Download video subtitles | **Any for free**: single video, playlist, user/channel, album, etc| Free for single video |
| The number of subtitles you can download for a video | **All of them** | Up to 10 |
| Convert subtitles to additional formats | **Yes** | No |
| Support LRC format | **Yes** | No |
| Select audio codec for audio/video | **Yes** | No |
| Extract and convert additional audio tracks for video | **Yes** | No |
| Simultaneous downloads | **Unlimited** | 1 free, 3 for 12 EUR, 7 for 43 EUR|
| Private YouTube content download | **Free** | Only in paid plans *starts from 12 EUR* |
| **Paid** | **No** | Yes |
| **Free options** | **The program is completely free** | Only **30** videos per day, 5 from a channel, 10 from a playlist |
| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
| Posts and Captions Export | No | Paid (43 EUR) |
| Advertisements free | **No ADs at all for free** | Paid (43 EUR) |
| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
| Select want content type to download | **Yes** | No |
| Instagram support | **Yes** | No |
| Twitter support | **Yes** | No |
| Reddit support | **Yes** | No |
| Other sites support | **Yes** | No |
| Still supported | Yes | Yes |
# 4K Stogram
https://www.4kdownload.com/products/product-stogram
<!---https://www.4kdownload.com/products/product-stogram--->
https://www.4kdownload.com/-ad0p9/stogram
| Option | SCrawler | 4K Stogram |
| ---- | ---- | ---- |
@@ -18,7 +75,7 @@ https://www.4kdownload.com/products/product-stogram
| Download posts by location | No | **Yes** |
| Save Private Instagram Content with Permission| Yes | Yes |
| Download Instagram Stories and Highlights | Yes | Yes |
| See Others Instagram Feed As Your Own | No | **Yes** |
| See Others Instagram Feed As Your Own | Yes | Yes |
| Download Instagram Video Posts | Yes | Yes |
| Backup Your Instagram Account | Yes | Yes |
| Save Instagram Posts by Date | Yes | Yes |
@@ -27,11 +84,11 @@ https://www.4kdownload.com/products/product-stogram
| Export and import subscriptions | No | **Yes** |
| **Paid** | **No** | Yes |
| **Free options** | **The program is completely free** | Only **ONE** profile downloading and only **200 posts** per day |
| Permitted Commercial Use | **Yes** | Starting from 43.56 EUR |
| Automatic Subscriptions Update | **Free** | Paid (43.56 EUR) |
| Posts and Captions Export | No | Paid (43.56 EUR) |
| Advertisements free | **No ADs at all for free** | Paid (14.52) |
| Operating Systems | Windows 7+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
| Permitted Commercial Use | **Yes** | Starting from 43 EUR |
| Automatic Subscriptions Update | **Free** | Paid (43 EUR) |
| Posts and Captions Export | No | Paid (43 EUR) |
| Advertisements free | **No ADs at all for free** | Paid (18 EUR) |
| Operating Systems | Windows 10+ | Windows 7+, MacOS 10.13+, Ubuntu x64 |
| Select want content type to download | **Yes** | No |
| Instagram support | Yes | Yes |
| Twitter support | **Yes** | No |
@@ -66,17 +123,9 @@ https://github.com/RipMeApp/ripme
| Export and import subscriptions | No | No |
| **Paid** | **No** | **No** |
| **Free options** | The program is completely free | The program is completely free, but site limits are not declared |
| Operating Systems | Windows 7+ | Windows, MacOS, Linux |
| Operating Systems | Windows 10+ | Windows, MacOS, Linux |
| Select want content type to download | Yes | Yes |
| Suported sites | 3 internal and any site using plugins | 86+ sites (declared) |
| Suported sites | 15+ internal and any site using plugins | 86+ sites (declared) |
| Other sites support | **Yes** | No |
| Still supported | **Yes** | **No (last release date May 4, 2021)** |
# gallery-dl
https://github.com/mikf/gallery-dl
**CLI tool**! Configured with JSON files only. Users need to learn complex configuration options, JSON, commands to use that tool. Very difficult to configure.
SCrawler has advanced user management, collections, labels, groups, automatic downloads, a beautiful view, GUI, the ability to add plugins for other sites and much more. Just try it and compare.

225
README.md
View File

@@ -1,151 +1,230 @@
# :rainbow_flag: Social networks crawler :rainbow_flag:
<!-- # :rainbow_flag: Happy LGBT Pride Month :tada:
-->
# 🏳️‍🌈 Social networks crawler 🏳️‍🌈
[![GitHub release (latest by date)](https://img.shields.io/github/v/release/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/releases/latest)
[![GitHub license](https://img.shields.io/github/license/AAndyProgram/SCrawler)](https://github.com/AAndyProgram/SCrawler/blob/main/LICENSE)
[![Discord](https://img.shields.io/discord/1124032649682493462?logo=discord)](https://discord.gg/uFNUXvFFmg)
[![GitHub all releases](https://img.shields.io/github/downloads/aandyprogram/scrawler/total?label=Total%20downloads)](https://github.com/AAndyProgram/SCrawler/releases)
[![FAQ](https://img.shields.io/badge/FAQ-green)](FAQ.md)
[![GUIDE](https://img.shields.io/badge/GUIDE-green)](https://github.com/AAndyProgram/SCrawler/wiki)
[![How to support](https://img.shields.io/badge/HowToSupport-green)](HowToSupport.md)
:eu:
:greece:
A program to download photo and video from [any site](#supported-sites) (e.g. Reddit, Twitter, Instagram).
A program to download photo and video from [any site](#supported-sites) (e.g. YouTube, YouTube Music, OnlyFans, Reddit, Twitter, Mastodon, Instagram, Threads, Facebook, TikTok, RedGifs, JustForFans, PornHub, XHamster, XVIDEOS, ThisVid, LPSG, Pinterest).
Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
**If you like SCrawler, please like the program on [this site](https://alternativeto.net/software/scrawler/about/) and/or [this](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml)**
[![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)
**Join our Discord server**: https://discord.gg/uFNUXvFFmg
<br/>*If you have problems using the program, you can get help faster on our Discord server!*
**Bitcoin**: bitcoin:BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
<!---Do you like this program? Consider adding to my coffee fund by making a donation to show your support. :blush:
[![ko-fi](https://www.ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/andyprogram)--->
**Bitcoin**: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET
[![](https://www.softpedia.com/_img/softpedia_100_free.png?2023_1)](https://www.softpedia.com/get/Internet/Download-Managers/Social-networks-crawler.shtml#status)
![Main window](ProgramScreenshots/MainWindow.png)
![Channels window](ProgramScreenshots/Channels.png)
[**YouTube standalone application:**](https://github.com/AAndyProgram/SCrawler/wiki/YouTube-downloader)
![YouTube application](ProgramScreenshots/AppYouTube.png)
# What can program do:
- Download pictures and videos from users' profiles and subreddits:
- Reddit images;
- Reddit galleries of images;
- Reddit videos (downloading Reddit hosted video is going through ffmpeg (**ffmpeg only works with the x64 program**));
- YouTube videos, shorts, community feeds, users, artists, playlists, music, tracks;
- Reddit images, galleries of images, videos, saved posts;
- Redgifs videos (https://www.redgifs.com/);
- Twitter images and videos;
- Instagram images and videos;
- Instagram tagged posts;
- Instagram stories;
- Twitter images and videos, saved (bookmarked) posts;
- OnlyFans images and videos, saved (bookmarked) posts;
- JustForFans images and videos, saved (bookmarked) posts;
- Mastodon images and videos, saved (bookmarked) posts;
- Instagram images and videos, tagged posts, stories, saved posts;
- Threads images and videos;
- Facebook images and videos, stories, saved posts;
- TikTok videos;
- Pinterest boards, users, saved posts;
- Imgur images, galleries and videos;
- Gfycat videos;
- PornHub images, videos, save (liked) posts, search queries, search categories;
- XHamster images, videos, saved posts, search queries, search categories, search tags;
- XVIDEOS videos, saved posts, search queries, search categories;
- ThisVid images, videos, saved posts, search queries, search categories, search tags;
- [Other](#supported-sites) supported sites
- Parse [channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
- Download [saved Reddit, Twitter and Instagram posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts)
- Parse [Reddit channel and view data](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
- Download [saved posts](https://github.com/AAndyProgram/SCrawler/wiki/Home#saved-posts)
- Add users from parsed channel
- **Advanced user management**
- **Automation** (downloading data automatically every ```X``` minutes)
- **Automation** ([downloading data automatically](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation) every ```X``` minutes)
- **Feed** ([feed](https://github.com/AAndyProgram/SCrawler/wiki#feed) of downloaded media files and subscriptions posts)
- Multiple accounts support
- Labeling users
- Create download groups
- Create [download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
- Adding users to favorites and temporary
- Filter exists users by label or group
- Adding users and search queries in the **Subscription** mode (download post preview, but do not download the media file)
- [Filter exists users](https://github.com/AAndyProgram/SCrawler/wiki#view) by label or group
- Selection of media types you want to download (images only, videos only, both)
- Download a special video, image or gallery
- Making collections (grouping users into collections)
- [Download a special video](https://github.com/AAndyProgram/SCrawler/wiki#download-separate-video), image or gallery
- Making [collections](https://github.com/AAndyProgram/SCrawler/wiki#collections) (grouping users into collections)
- Specifying a user folder (for downloading data to another location)
- Changing user icons
- Changing view modes
- Changing [view modes](https://github.com/AAndyProgram/SCrawler/wiki#view)
- ...and many others...
# Supported sites
- **YouTube**
- **YouTube Music**
- **Reddit**
- **Twitter**
- **OnlyFans** *(partial support)*[^1]
- **Mastodon**
- **Instagram**
- **Threads**
- **Facebook**
- JustForFans *(partial support)*[^1]
- TikTok
- RedGifs
- Pinterest
- Imgur
- Gfycat
- LPSG
- XVIDEOS
- **PornHub**
- **XHamster**
- **XVIDEOS**
- **ThisVid**
- [Other sites](Plugins.md)
# How does it works:
**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
# How it works
First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.
## Reddit
The program parses all user posts, obtain MD5 images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.
## Other sites
The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.
You can read about Instagram restrictions [here](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram-limits)
## How to request a new site
Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about
<!---Read [here](CONTRIBUTING.md#how-to-request-a-new-site) about--->
**I'm currently not accepting requests to develop new sites.**
# Requirements
- Windows 10, 11 with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this [tool](Tools/NET.FrameworkVersion.ps1).
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for Twitter (if you want to download data from Twitter)
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) for Instagram (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for saved Instagram posts, Instagram [stories authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-stories-authorization-headers) for Stories and Tagged data
- ffmpeg library for downloading videos hosted on Reddit (you can download it from the [official repo](https://github.com/GyanD/codexffmpeg/releases/tag/2021-01-12-git-ca21cb1e36) or [from my first release](https://github.com/AAndyProgram/SCrawler/releases/download/1.0.0.0/ffmpeg.zip)). **ffmpeg only works with the x64 version of the program.**
- **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
# Guide
- [Main window](https://github.com/AAndyProgram/SCrawler/wiki)
- [Users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
- [Add/Edit/Delete users](https://github.com/AAndyProgram/SCrawler/wiki/Users)
- [Collections](https://github.com/AAndyProgram/SCrawler/wiki#collections)
- [User operations](https://github.com/AAndyProgram/SCrawler/wiki#context-menu)
- [User labels](https://github.com/AAndyProgram/SCrawler/wiki/Users#labels)
- **[DOWNLOAD](https://github.com/AAndyProgram/SCrawler/wiki#download)**
- [Automation](https://github.com/AAndyProgram/SCrawler/wiki/Settings#automation)
- [Download groups](https://github.com/AAndyProgram/SCrawler/wiki/Settings#download-groups)
- [Downloading information](https://github.com/AAndyProgram/SCrawler/wiki#info)
- [Reddit channels](https://github.com/AAndyProgram/SCrawler/wiki/Channels)
- [Saved posts](https://github.com/AAndyProgram/SCrawler/wiki#saved-posts)
- [View modes, filters](https://github.com/AAndyProgram/SCrawler/wiki#view)
- **[SETTINGS](https://github.com/AAndyProgram/SCrawler/wiki/Settings)**
- **[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
- [Reddit](https://github.com/AAndyProgram/SCrawler/wiki/Settings#reddit)
- [Twitter](https://github.com/AAndyProgram/SCrawler/wiki/Settings#twitter)
- [OnlyFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#onlyfans)
- [Mastodon](https://github.com/AAndyProgram/SCrawler/wiki/Settings#mastodon)
- [Instagram](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram)
- [Threads](https://github.com/AAndyProgram/SCrawler/wiki/Settings#threads)
- [Facebook](https://github.com/AAndyProgram/SCrawler/wiki/Settings#facebook)
- [JustForFans](https://github.com/AAndyProgram/SCrawler/wiki/Settings#justforfans)
- [TikTok](https://github.com/AAndyProgram/SCrawler/wiki/Settings#tiktok)
- [RedGifs](https://github.com/AAndyProgram/SCrawler/wiki/Settings#redgifs)
- [YouTube](https://github.com/AAndyProgram/SCrawler/wiki/Settings#youtube)
- [Pinterest](https://github.com/AAndyProgram/SCrawler/wiki/Settings#Pinterest)
- [PornHub](https://github.com/AAndyProgram/SCrawler/wiki/Settings#pornhub)
- [XHamster](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xhamster)
- [XVIDEOS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#xvideos)
- [ThisVid](https://github.com/AAndyProgram/SCrawler/wiki/Settings#thisvid)
- [LPSG](https://github.com/AAndyProgram/SCrawler/wiki/Settings#lpsg)
**Full guide you can find [here](https://github.com/AAndyProgram/SCrawler/wiki)**
**Video on how to configure**
[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
# Installation
**Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder, copy the file ```ffmpeg.exe``` into it and enjoy.** :blush:
**Just download the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest), unzip the program archive to any folder and enjoy.** :blush:
**Don't put program in the ```Program Files``` system folder (this is portable program and program settings are stored in the program folder)**
**I highly doubt you can run SCrawler on Linux or Mac. SCrawler is a program that is heavily dependent on Windows.**
# Updating
Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. **Before starting a new version, I recommend making a backup copy of the program settings folder.**
Just download [latest](https://github.com/AAndyProgram/SCrawler/releases/latest) version and unpack it into the program folder. **Before launching a new version, I recommend making a backup copy of the program settings folder and user settings/data files.**
# How to build from source
**You can also use the updater included in the release package.**
1. Delete the "PersonalUtilities" project from the solution.
1. Delete the "PersonalUtilities.Notifications" project from the solution.
1. Add the latest versions of the ```PersonalUtilities.dll``` and ```PersonalUtilities.Notifications.dll``` libraries (from the [latest release](https://github.com/AAndyProgram/SCrawler/releases/latest)).
1. Import PersonalUtilities.Functions for the whole project.
# [How to report a problem](CONTRIBUTING.md#how-to-report-a-problem)
# How to make a plugin
# [How to build from source](CONTRIBUTING.md#how-to-build-from-source)
Read about how to make plugin [here](https://github.com/AAndyProgram/SCrawler/wiki/Plugins).
# [How to make a plugin](https://github.com/AAndyProgram/SCrawler/wiki/Plugins)
# How to support
Read more about how to support the program [here](HowToSupport.md).
# [How to support](HowToSupport.md)
# Settings and usage
The program has an intuitive interface.
You need to set up authorization for Twitter and Instagram:
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies) and [tokens](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-twitter-tokens) for **Twitter** (if you want to download data from Twitter)
- Authorization [cookies](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-set-up-cookies), [Hash](https://github.com/AAndyProgram/SCrawler/wiki/Settings#instagram) and [authorization headers](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-authorization-headers) for **Instagram** (if you want to download data from Instagram), [Hash 2](https://github.com/AAndyProgram/SCrawler/wiki/Settings#how-to-find-instagram-hash-2) for **saved Instagram posts**
**[SITES REQUIREMENTS](https://github.com/AAndyProgram/SCrawler/wiki/Settings#sites-requirements)**
Just add a user profile and **click the ```Start downloading``` button**.
[![How to configure](https://img.youtube.com/vi/XDn7zG4I700/0.jpg)](https://www.youtube.com/watch?v=XDn7zG4I700)
You can add users by patterns:
- https://www.instagram.com/SomeUserName
- https://twitter.com/SomeUserName
- https://reddit.com/user/SomeUserName
- https://reddit.com/r/SomeSubredditName
- https://www.redgifs.com/users/SomeUserName
- u/SomeUserName
- r/SomeSubredditName
- SomeUserName (in this case, you need to select the user's site)
- SomeSubredditName
Just add a user profile and **click the ```Download``` button**.
Read more about adding users and subreddits [here](https://github.com/AAndyProgram/SCrawler/wiki/Users)
```mermaid
stateDiagram
Start: Add site credentials
What: What would I like to do
DownUser: Download user
DownVideo: Download video
AUser: Add user (1)
OVIF: Open standalone downloader (2)
AVideo: Add video url
F5: Press 'F5' or click the download button
[*]-->Start
Start-->What
What-->DownUser
What-->DownVideo
DownUser-->AUser
DownVideo-->OVIF
OVIF-->AVideo
AVideo-->F5
AUser-->F5
F5-->[*]
```
1. Press `Insert` or click the `Download` button ([read more here](https://github.com/AAndyProgram/SCrawler/wiki#users-list), [hot keys](https://github.com/AAndyProgram/SCrawler/wiki#hot-keys))
2. Click the `Download` button, then `Standalone downloader` ([read more here](https://github.com/AAndyProgram/SCrawler/wiki#download-separate-video))
![Add user](ProgramScreenshots/CreateUserClear.png)
# Using program as just video downloader
Create a shortcut for the program. Open shortcut properties. In the ```Shortcut``` tab, in the ```Target``` field, just add the letter ```v``` at the end across the space.
Example: ```D:\Programs\SCrawler\SCrawler.exe v```
![Separate video downloader](ProgramScreenshots/SeparateVideoDownloader.png)
# Contact me
[![matrix](https://img.shields.io/badge/Matrix-%40andyprogram%3Amatrix.org-informational)](https://matrix.to/#/@andyprogram:matrix.org)
Discord server: https://discord.gg/uFNUXvFFmg
<!--
[e-mail](mailto:andyprogram@proton.me): andyprogram@proton.me
Matrix (Element): https://matrix.to/#/@andyprogram:matrix.org
Discord (contact the developer): andyprogram
Discord server: https://discord.gg/uFNUXvFFmg
[Wire](https://account.wire.com/user-profile/?id=93985052-cf2c-4b72-ac75-bbe3231cf544): @andyprogram
-->
[^1]: Partial support means that I don't have personal accounts on paid porn sites because I don't pay for porn. If this site has stopped downloading and you want me to fix it, please be ready to give me access to an account with at least one active subscription. Otherwise, the download from this site will not be fixed.

View File

@@ -30,19 +30,5 @@ Friend Module Declarations
Friend ReadOnly Property FileRegExExt As New RParams(FileUrlRegexDefault, 0, Nothing, InputForbidRemover)
Friend ReadOnly Property FileRegExExt2 As New RParams("([^/]+?)(?=(\Z|&))", 0, Nothing, InputForbidRemover)
Friend ReadOnly Property FileExistsRegEx As RParams = RParams.DMS(FileUrlRegexDefault, 2)
Private Class PUMComparer : Implements IEqualityComparer, IEqualityComparer(Of PluginUserMedia)
Private Overloads Function Equals(ByVal x As PluginUserMedia, ByVal y As PluginUserMedia) As Boolean Implements IEqualityComparer(Of PluginUserMedia).Equals
Return x.URL = y.URL
End Function
Private Function IEqualityComparer_Equals(ByVal x As Object, ByVal y As Object) As Boolean Implements IEqualityComparer.Equals
Return DirectCast(x, PluginUserMedia).URL = DirectCast(y, PluginUserMedia).URL
End Function
Private Overloads Function GetHashCode(ByVal Obj As Object) As Integer Implements IEqualityComparer.GetHashCode
Throw New NotImplementedException()
End Function
Private Overloads Function GetHashCode(ByVal Obj As PluginUserMedia) As Integer Implements IEqualityComparer(Of PluginUserMedia).GetHashCode
Throw New NotImplementedException()
End Function
End Class
Friend ReadOnly TempListAddParams As New ListAddParams(LAP.NotContainsOnly) With {.Comparer = New PUMComparer}
Friend ReadOnly TempListAddParams As New ListAddParams(LAP.NotContainsOnly) With {.Comparer = New FComparer(Of PluginUserMedia)(Function(x, y) x.URL = y.URL)}
End Module

View File

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
<Assembly: AssemblyDescription("LPSG plugin for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("LPSG")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2022.7.7.0")>
<Assembly: AssemblyFileVersion("2022.7.7.0")>
<Assembly: AssemblyVersion("2022.10.12.0")>
<Assembly: AssemblyFileVersion("2022.10.12.0")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -30,7 +30,7 @@ Public Class SiteSettings : Implements ISiteSettings
.LoadSettings()
Else
.CookiesDomain = "www.lpsg.com"
.Cookies = New CookieKeeper("www.lpsg.com")
.Cookies = New CookieKeeper(.CookiesDomain)
End If
End With
End Sub
@@ -54,6 +54,10 @@ Public Class SiteSettings : Implements ISiteSettings
End Sub
#End Region
#Region "Update"
Public Sub BeginEdit() Implements ISiteSettings.BeginEdit
End Sub
Public Sub EndEdit() Implements ISiteSettings.EndEdit
End Sub
Public Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
End Sub
Public Sub EndUpdate() Implements ISiteSettings.EndUpdate
@@ -81,7 +85,7 @@ Public Class SiteSettings : Implements ISiteSettings
Else
Return Nothing
End If
Catch ex As Exception
Catch
Return Nothing
End Try
End Function
@@ -92,9 +96,12 @@ Public Class SiteSettings : Implements ISiteSettings
Return Nothing
End Function
Public Function Available(ByVal What As ISiteSettings.Download, ByVal Silent As Boolean) As Boolean Implements ISiteSettings.Available
Return True
Return If(Responser.Cookies?.Count, 0) > 0
End Function
Public Function ReadyToDownload(ByVal What As ISiteSettings.Download) As Boolean Implements ISiteSettings.ReadyToDownload
Return True
End Function
Public Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl
Return String.Empty
End Function
End Class

View File

@@ -38,7 +38,8 @@ Public Class UserData : Implements IPluginContentProvider
Public Property SeparateVideoFolder As Boolean Implements IPluginContentProvider.SeparateVideoFolder
Public Property DataPath As String Implements IPluginContentProvider.DataPath
Public Property PostsNumberLimit As Integer? Implements IPluginContentProvider.PostsNumberLimit
Public Property PostsDateLimit As Date? Implements IPluginContentProvider.PostsDateLimit
Public Property DownloadDateFrom As Date? Implements IPluginContentProvider.DownloadDateFrom
Public Property DownloadDateTo As Date? Implements IPluginContentProvider.DownloadDateTo
#End Region
#Region "Interface exchange options"
Public Sub ExchangeOptionsSet(ByVal Obj As Object) Implements IPluginContentProvider.ExchangeOptionsSet
@@ -95,7 +96,7 @@ Public Class UserData : Implements IPluginContentProvider
If Responser.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
LogProvider.Add("LPSG not available")
Else
LogProvider.Add(ex, "[LPSG.UserData.GetMedia]")
LogProvider.Add(ex, $"[LPSG.UserData.GetMedia({Name})]")
End If
End Try
End Sub
@@ -152,7 +153,8 @@ Public Class UserData : Implements IPluginContentProvider
If Responser.Client.StatusCode = Net.HttpStatusCode.ServiceUnavailable Then
LogProvider.Add("LPSG not available")
Else
m.DownloadState = UStates.Skipped
m.DownloadState = UStates.Missing
m.Attempts += 1
End If
End Try
RaiseEvent ProgressChanged(1)

View File

@@ -47,8 +47,7 @@ Friend NotInheritable Class M3U8
CachePath.Delete(SFO.Path, SFODelete.None, EDP.None)
End Try
End Function
Friend Shared Function Download(ByVal URL As String, ByVal Appender As String, ByVal ffmpegFile As SFile, ByVal f As SFile,
ByRef Logger As ILogProvider) As SFile
Friend Shared Function Download(ByVal URL As String, ByVal Appender As String, ByVal ffmpegFile As SFile, ByVal f As SFile, ByRef Logger As ILogProvider) As SFile
Try
If Not URL.IsEmptyString Then
Using w As New WebClient
@@ -62,7 +61,7 @@ Friend NotInheritable Class M3U8
End If
Return Nothing
Catch ex As Exception
If Not ex.HelpLink = 1 Then Logger.Add(ex, "[M3U8.Download]")
If Not ex.HelpLink = 1 Then Logger.Add(ex, $"[M3U8.Download({URL}, {Appender}, {ffmpegFile}, {f})]")
Throw ex
End Try
End Function

View File

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
<Assembly: AssemblyDescription("XVIDEOS plugin for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("XVIDEOS")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2022.7.7.0")>
<Assembly: AssemblyFileVersion("2022.7.7.0")>
<Assembly: AssemblyVersion("2022.10.12.0")>
<Assembly: AssemblyFileVersion("2022.10.12.0")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -17,19 +17,15 @@ Public Class SettingsForm
MyDefs = New DefaultFormOptions(Me, Design)
End Sub
Private Sub SettingsForm_Load(sender As Object, e As EventArgs) Handles Me.Load
Try
With MyDefs
.MyViewInitialize(True)
.AddEditToolbar({EditToolbar.ControlItem.Add, EditToolbar.ControlItem.Delete})
.AddOkCancelToolbar()
If Settings.Domains.Count > 0 Then Settings.Domains.ForEach(Sub(d) LIST_DOMAINS.Items.Add(d))
.EndLoaderOperations()
End With
Catch ex As Exception
MyDefs.InvokeLoaderError(ex)
End Try
With MyDefs
.MyViewInitialize(True)
.AddEditToolbar({EditToolbar.ControlItem.Add, EditToolbar.ControlItem.Delete})
.AddOkCancelToolbar()
If Settings.Domains.Count > 0 Then Settings.Domains.ForEach(Sub(d) LIST_DOMAINS.Items.Add(d))
.EndLoaderOperations()
End With
End Sub
Private Sub MyDefs_ButtonOkClick() Handles MyDefs.ButtonOkClick
Private Sub MyDefs_ButtonOkClick(ByVal Sender As Object, ByVal e As KeyHandleEventArgs) Handles MyDefs.ButtonOkClick
Settings.Domains.Clear()
With LIST_DOMAINS
If .Items.Count > 0 Then
@@ -39,7 +35,7 @@ Public Class SettingsForm
Settings.UpdateDomains()
MyDefs.CloseForm()
End Sub
Private Sub MyDefs_ButtonAddClick() Handles MyDefs.ButtonAddClick
Private Sub MyDefs_ButtonAddClick(ByVal Sender As Object, ByVal e As EditToolbarEventArgs) Handles MyDefs.ButtonAddClick
Dim nd$ = InputBoxE("Enter a new domain using the pattern [xvideos.com]:", "New domain")
If Not nd.IsEmptyString Then
If Not LIST_DOMAINS.Items.Contains(nd) Then
@@ -49,11 +45,10 @@ Public Class SettingsForm
End If
End If
End Sub
Private Sub MyDefs_ButtonDeleteClick() Handles MyDefs.ButtonDeleteClickE
Private Sub MyDefs_ButtonDeleteClickE(ByVal Sender As Object, ByVal e As EditToolbarEventArgs) Handles MyDefs.ButtonDeleteClickE
If _LatestSelected.ValueBetween(0, LIST_DOMAINS.Items.Count - 1) Then
Dim n$ = LIST_DOMAINS.Items(_LatestSelected)
If MsgBoxE({$"Are you sure you want to delete the [{n}] domain?",
"Removing domains"}, MsgBoxStyle.YesNo) = MsgBoxResult.Yes Then
If MsgBoxE({$"Are you sure you want to delete the [{n}] domain?", "Removing domains"}, MsgBoxStyle.YesNo) = MsgBoxResult.Yes Then
LIST_DOMAINS.Items.RemoveAt(_LatestSelected)
MsgBoxE($"Domain [{n}] removed")
Else

View File

@@ -26,7 +26,7 @@ Public Class SiteSettings : Implements ISiteSettings
Public Property Logger As ILogProvider Implements ISiteSettings.Logger
#Region "M3U8"
Private ReadOnly OS64 As Boolean
Private ReadOnly FfmpegExists As Boolean
Friend ReadOnly FfmpegExists As Boolean
Friend ReadOnly FfmpegFile As SFile
Friend ReadOnly Property UseM3U8 As Boolean
Get
@@ -97,6 +97,10 @@ Public Class SiteSettings : Implements ISiteSettings
Public Sub DownloadDone(ByVal What As ISiteSettings.Download) Implements ISiteSettings.DownloadDone
End Sub
#End Region
Public Sub BeginEdit() Implements ISiteSettings.BeginEdit
End Sub
Public Sub EndEdit() Implements ISiteSettings.EndEdit
End Sub
Public Sub BeginUpdate() Implements ISiteSettings.BeginUpdate
End Sub
Public Sub EndUpdate() Implements ISiteSettings.EndUpdate
@@ -179,4 +183,7 @@ Public Class SiteSettings : Implements ISiteSettings
End If
Return Nothing
End Function
Public Function GetUserPostUrl(ByVal UserID As String, ByVal PostID As String) As String Implements ISiteSettings.GetUserPostUrl
Return String.Empty
End Function
End Class

View File

@@ -14,8 +14,8 @@ Imports UStates = SCrawler.Plugin.PluginUserMedia.States
Imports UTypes = SCrawler.Plugin.PluginUserMedia.Types
Public Class UserData : Implements IPluginContentProvider
#Region "Interface declarations"
Public Event ProgressChanged(Count As Integer) Implements IPluginContentProvider.ProgressChanged
Public Event TotalCountChanged(Count As Integer) Implements IPluginContentProvider.TotalCountChanged
Public Event ProgressChanged(ByVal Count As Integer) Implements IPluginContentProvider.ProgressChanged
Public Event TotalCountChanged(ByVal Count As Integer) Implements IPluginContentProvider.TotalCountChanged
Public Property Thrower As IThrower Implements IPluginContentProvider.Thrower
Public Property LogProvider As ILogProvider Implements IPluginContentProvider.LogProvider
Public Property ESettings As ISiteSettings Implements IPluginContentProvider.Settings
@@ -37,7 +37,8 @@ Public Class UserData : Implements IPluginContentProvider
Public Property SeparateVideoFolder As Boolean Implements IPluginContentProvider.SeparateVideoFolder
Public Property DataPath As String Implements IPluginContentProvider.DataPath
Public Property PostsNumberLimit As Integer? Implements IPluginContentProvider.PostsNumberLimit
Public Property PostsDateLimit As Date? Implements IPluginContentProvider.PostsDateLimit
Public Property DownloadDateFrom As Date? Implements IPluginContentProvider.DownloadDateFrom
Public Property DownloadDateTo As Date? Implements IPluginContentProvider.DownloadDateTo
#End Region
#Region "Interface exchange options"
Public Sub ExchangeOptionsSet(ByVal Obj As Object) Implements IPluginContentProvider.ExchangeOptionsSet
@@ -56,14 +57,21 @@ Public Class UserData : Implements IPluginContentProvider
Private Property Responser As Response
Public Sub GetMedia() Implements IPluginContentProvider.GetMedia
Try
If Not Settings.UseM3U8 Then LogProvider.Add("File [ffmpeg.exe] not found") : Exit Sub
If Not Settings.UseM3U8 Then
If Settings.FfmpegExists Then
LogProvider.Add($"XVIDEOS [{Name}]: The plugin only works with x64 OS.")
Else
LogProvider.Add($"XVIDEOS [{Name}]: File [ffmpeg.exe] not found")
End If
Exit Sub
End If
If Not Responser Is Nothing Then Responser.Dispose()
Responser = New Response
Responser.Copy(Settings.Responser)
Dim NextPage% = 0
Dim r$
Dim j As EContainer, jj As EContainer
Dim jj As EContainer
Dim e As ErrorsDescriber = EDP.ThrowException
Dim user$ = Settings.GetUserUrl(Name, False)
Dim p As PluginUserMedia
@@ -74,8 +82,7 @@ Public Class UserData : Implements IPluginContentProvider
r = Responser.GetResponse($"https://www.xvideos.com/{user}/videos/new/{If(NextPage = 0, String.Empty, NextPage)}",, e)
If Not r.IsEmptyString Then
If Not EnvirSet Then UserExists = True : UserSuspended = False : EnvirSet = True
j = JsonDocument.Parse(r).XmlIfNothing
With j
With JsonDocument.Parse(r).XmlIfNothing
If .Contains("videos") Then
With .Item("videos")
If .Count > 0 Then
@@ -86,9 +93,12 @@ Public Class UserData : Implements IPluginContentProvider
.URL = $"https://www.xvideos.com{jj.Value("u")}"
}
If Not p.PostID.IsEmptyString And Not jj.Value("u").IsEmptyString Then
If Not TempPostsList.Contains(p.PostID) Then TempPostsList.Add(p.PostID) : TempMediaList.Add(p) Else Exit Do
If Not TempPostsList.Contains(p.PostID) Then TempPostsList.Add(p.PostID) : TempMediaList.Add(p) Else .Dispose() : Exit Do
End If
Next
Else
.Dispose()
Exit Do
End If
End With
Else
@@ -105,9 +115,7 @@ Public Class UserData : Implements IPluginContentProvider
If TempMediaList.Count > 0 Then
For i% = 0 To TempMediaList.Count - 1
Thrower.ThrowAny()
With TempMediaList(i)
TempMediaList(i) = GetVideoData(.URL, Responser, Settings.DownloadUHD.Value, .PostID, LogProvider)
End With
With TempMediaList(i) : TempMediaList(i) = GetVideoData(.URL, Responser, Settings.DownloadUHD.Value, .PostID, LogProvider) : End With
Next
TempMediaList.RemoveAll(Function(m) m.URL.IsEmptyString)
End If
@@ -149,7 +157,7 @@ Public Class UserData : Implements IPluginContentProvider
Dim t$ = RegexReplace(r, VideoTitleRegex)
r = resp.GetResponse(m,, EDP.ThrowException)
If Not r.IsEmptyString Then
Dim ls As List(Of VSize) = FNF.RegexFields(Of VSize)(r, {M3U8Reparse}, {1, 2})
Dim ls As List(Of VSize) = RegexFields(Of VSize)(r, {M3U8Reparse}, {1, 2})
If ls.ListExists And Not DownloadUHD Then ls.RemoveAll(Function(v) v.Size > 1080)
If ls.ListExists Then
ls.Sort()
@@ -203,7 +211,8 @@ Public Class UserData : Implements IPluginContentProvider
m.File = f
m.DownloadState = UStates.Downloaded
Catch ex As Exception
m.DownloadState = UStates.Skipped
m.DownloadState = UStates.Missing
m.Attempts += 1
End Try
TempMediaList(i) = m
RaiseEvent ProgressChanged(1)

View File

@@ -1,3 +1,3 @@
[*.vb]
# Modifier preferences
file_header_template = Copyright (C) 2022 Andy\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>
file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -38,34 +38,56 @@ Namespace Plugin.Attributes
Public Property LabelTextAlign As Drawing.ContentAlignment = Drawing.ContentAlignment.TopCenter
''' <summary>This is an authorization property</summary>
Public Property IsAuth As Boolean = False
Public Property Category As String = Nothing
Public Property InheritanceName As String = Nothing
''' <summary>Initialize a new property option attribute</summary>
''' <param name="PropertyName">Property name</param>
Public Sub New(<CallerMemberName()> Optional ByVal PropertyName As String = Nothing)
Name = PropertyName
End Sub
End Class
''' <summary>Set the dependent fields that need to be updated when this property is changed internally.</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class DependentFields : Inherits Attribute
Public ReadOnly Fields As String()
Public Sub New(ByVal Field As String)
Fields = {Field}
End Sub
Public Sub New(ByVal Fields As String())
Me.Fields = Fields
End Sub
End Class
''' <summary>Store property value in settings XML file</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class PXML : Inherits Attribute
Public ReadOnly ElementName As String
Public Property OnlyForChecked As Boolean = False
''' <summary>Initialize a new XML attribute</summary>
''' <param name="XMLElementName">XML element name</param>
Public Sub New(<CallerMemberName()> Optional ByVal XMLElementName As String = Nothing)
ElementName = XMLElementName
End Sub
End Class
''' <summary>Attribute to disable some properties for host use</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class DoNotUse : Inherits Attribute
Public ReadOnly Value As Boolean = True
Public Sub New()
End Sub
Public Sub New(ByVal Value As Boolean)
Me.Value = Value
End Sub
End Class
''' <summary>Special property updater</summary>
<AttributeUsage(AttributeTargets.Method, AllowMultiple:=True, Inherited:=False)> Public NotInheritable Class PropertyUpdater : Inherits Attribute
Public ReadOnly Name As String
Public ReadOnly Dependencies As String()
Public ReadOnly Arguments As String()
''' <inheritdoc cref="PropertyUpdater.New(String, String())"/>
Public Sub New(ByVal UpdatingPropertyName As String)
Name = UpdatingPropertyName
End Sub
''' <summary>Initialize a new PropertyUpdater attribute</summary>
''' <param name="UpdatingPropertyName">The name of the property to be updated</param>
Public Sub New(ByVal UpdatingPropertyName As String, ByVal Dependent As String())
Public Sub New(ByVal UpdatingPropertyName As String, ByVal Arguments As String())
Name = UpdatingPropertyName
Dependencies = Dependent
Me.Arguments = Arguments
End Sub
End Class
''' <summary>Plugin key</summary>
@@ -97,6 +119,8 @@ Namespace Plugin.Attributes
''' <see langword="False"/> - only for conversion
''' </summary>
Public FieldsChecker As Boolean = False
''' <summary>Interaction with changing text field. Default: <see langword="False"/></summary>
Public Interaction As Boolean = False
''' <summary>Initialize a new Provider attribute. <see cref="IFormatProvider"/> is only allowed</summary>
''' <param name="PropertyName">The name of the property for which this provider is used</param>
Public Sub New(ByVal PropertyName As String)
@@ -129,13 +153,26 @@ Namespace Plugin.Attributes
''' Predefined task counter.<br/>
''' <see cref="TaskCounter"/> will take precedence if it is defined.
''' </param>
Public Sub New(Optional ByVal JobsCount As Integer = -1)
TasksCount = JobsCount
Public Sub New(Optional ByVal TasksCount As Integer = -1)
Me.TasksCount = TasksCount
End Sub
End Class
''' <summary>A property attribute that specifies how many users should be downloaded at the same time in one thread</summary>
<AttributeUsage(AttributeTargets.Property, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class TaskCounter : Inherits Attribute
End Class
''' <remarks>
''' This attribute cannot be combined with <see cref="SeparatedTasks"/>.
''' If set to <see cref="SeparatedTasks"/>, this attribute will be ignored
''' </remarks>
''' <inheritdoc cref="SeparatedTasks"/>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class TaskGroup : Inherits Attribute
Public ReadOnly Name As String
''' <summary>Initialize a new TaskGroup attribute.</summary>
''' <param name="Name">Group name</param>
Public Sub New(ByVal Name As String)
Me.Name = Name
End Sub
End Class
''' <summary>This attribute indicates that the plugin has a SavedPosts environment</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class SavedPosts : Inherits Attribute
End Class
@@ -154,4 +191,13 @@ Namespace Plugin.Attributes
Repository = RepoName
End Sub
End Class
''' <summary>Replace internal plugin with the current one</summary>
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Public NotInheritable Class ReplaceInternalPluginAttribute : Inherits Attribute
Public ReadOnly SiteName As String
Public ReadOnly PluginKey As String
Public Sub New(ByVal PluginKey As String, Optional ByVal SiteName As String = Nothing)
Me.PluginKey = PluginKey
Me.SiteName = SiteName
End Sub
End Class
End Namespace

View File

@@ -0,0 +1,35 @@
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Interface IDownloadableMedia : Inherits IUserMedia, IDisposable
Event CheckedChange As EventHandler
Event ThumbnailChanged As EventHandler
Event StateChanged As EventHandler
ReadOnly Property SiteIcon As Drawing.Image
ReadOnly Property Site As String
ReadOnly Property SiteKey As String
Property AccountName As String
Property ThumbnailUrl As String
Property ThumbnailFile As String
Property Title As String
Property Size As Integer
Property Duration As TimeSpan
Property Progress As Object
ReadOnly Property HasError As Boolean
ReadOnly Property Exists As Boolean
Property Checked As Boolean
Property Instance As IPluginContentProvider
Sub Download(ByVal UseCookies As Boolean, ByVal Token As Threading.CancellationToken)
Sub Delete(ByVal RemoveFiles As Boolean)
Sub Load(ByVal File As String)
Sub Save()
Overloads Function ToString() As String
Overloads Function ToString(ByVal ForMediaItem As Boolean) As String
End Interface
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -8,30 +8,38 @@
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Interface IPluginContentProvider : Inherits IDisposable
Event ProgressChanged(ByVal Count As Integer)
Event TotalCountChanged(ByVal Count As Integer)
Event ProgressChanged(ByVal Value As Integer)
Event ProgressMaximumChanged(ByVal Value As Integer, ByVal Add As Boolean)
Event ProgressPreChanged As ProgressChangedEventHandler
Event ProgressPreMaximumChanged As ProgressMaximumChangedEventHandler
Property Thrower As IThrower
Property LogProvider As ILogProvider
Property Settings As ISiteSettings
Property AccountName As String
Property Name As String
Property ID As String
Property Options As String
Property ParseUserMediaOnly As Boolean
Property UserDescription As String
Property ExistingContentList As List(Of PluginUserMedia)
Property ExistingContentList As List(Of IUserMedia)
Property TempPostsList As List(Of String)
Property TempMediaList As List(Of PluginUserMedia)
Property TempMediaList As List(Of IUserMedia)
Property UserExists As Boolean
Property UserSuspended As Boolean
Property IsSavedPosts As Boolean
Property IsSubscription As Boolean
Property SeparateVideoFolder As Boolean
Property DataPath As String
Property PostsNumberLimit As Integer?
Property PostsDateLimit As Date?
Property DownloadDateFrom As Date?
Property DownloadDateTo As Date?
Function ExchangeOptionsGet() As Object
Sub ExchangeOptionsSet(ByVal Obj As Object)
Sub XmlFieldsSet(ByVal Fields As List(Of KeyValuePair(Of String, String)))
Function XmlFieldsGet() As List(Of KeyValuePair(Of String, String))
Sub GetMedia()
Sub Download()
Sub GetMedia(ByVal Token As Threading.CancellationToken)
Sub Download(ByVal Token As Threading.CancellationToken)
Sub DownloadSingleObject(ByVal Data As IDownloadableMedia, ByVal Token As Threading.CancellationToken)
Sub ResetHistoryData()
End Interface
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -8,31 +8,39 @@
' but WITHOUT ANY WARRANTY
Imports System.Drawing
Namespace Plugin
Public Interface ISiteSettings
Public Interface ISiteSettings : Inherits IDisposable
Enum Download As Integer
Main = 0
SavedPosts = 1
Channel = 2
SingleObject = 2
End Enum
ReadOnly Property Icon As Icon
ReadOnly Property Image As Image
ReadOnly Property Site As String
Property CMDEncoding As String
Property EnvironmentPrograms As IEnumerable(Of String)
Sub EnvironmentProgramsUpdated()
Property AccountName As String
Property Temporary As Boolean
Property DefaultInstance As ISiteSettings
ReadOnly Property SubscriptionsAllowed As Boolean
Property Logger As ILogProvider
Function GetUserUrl(ByVal UserName As String, ByVal Channel As Boolean) As String
Function GetUserUrl(ByVal User As IPluginContentProvider) As String
Function IsMyUser(ByVal UserURL As String) As ExchangeOptions
Function IsMyImageVideo(ByVal URL As String) As ExchangeOptions
Function GetSpecialData(ByVal URL As String, ByVal Path As String, ByVal AskForPath As Boolean) As IEnumerable(Of PluginUserMedia)
Function GetInstance(ByVal What As Download) As IPluginContentProvider
#Region "XML Support"
Sub Load(ByVal XMLValues As IEnumerable(Of KeyValuePair(Of String, String)))
#End Region
Function GetSingleMediaInstance(ByVal URL As String, ByVal OutputFile As String) As IDownloadableMedia
Function GetUserPostUrl(ByVal User As IPluginContentProvider, ByVal Media As IUserMedia) As String
#Region "Initialization"
Sub BeginInit()
Sub EndInit()
Sub BeginUpdate()
Sub EndUpdate()
Sub BeginEdit()
Sub EndEdit()
#End Region
#Region "Site availability"
Property AvailableText As String
Function Available(ByVal What As Download, ByVal Silent As Boolean) As Boolean
Function ReadyToDownload(ByVal What As Download) As Boolean
#End Region
@@ -42,7 +50,10 @@ Namespace Plugin
Sub AfterDownload(ByVal User As Object, ByVal What As Download)
Sub DownloadDone(ByVal What As Download)
#End Region
Sub Update()
Function Clone(ByVal Full As Boolean) As ISiteSettings
Sub Delete()
Overloads Sub Update()
Overloads Sub Update(ByVal Source As ISiteSettings)
Sub Reset()
Sub OpenSettingsForm()
Sub UserOptions(ByRef Options As Object, ByVal OpenForm As Boolean)

View File

@@ -13,7 +13,7 @@ Imports System.Runtime.InteropServices
<Assembly: AssemblyDescription("Plugin provider for SCrawler")>
<Assembly: AssemblyCompany("AndyProgram")>
<Assembly: AssemblyProduct("SCrawler.PluginProvider")>
<Assembly: AssemblyCopyright("Copyright © 2022")>
<Assembly: AssemblyCopyright("Copyright © 2024")>
<Assembly: AssemblyTrademark("AndyProgram")>
<Assembly: ComVisible(False)>
@@ -32,6 +32,6 @@ Imports System.Runtime.InteropServices
' by using the '*' as shown below:
' <Assembly: AssemblyVersion("1.0.*")>
<Assembly: AssemblyVersion("2022.7.7.0")>
<Assembly: AssemblyFileVersion("2022.7.7.0")>
<Assembly: AssemblyVersion("2024.4.13.0")>
<Assembly: AssemblyFileVersion("2024.4.13.0")>
<Assembly: NeutralResourcesLanguage("en")>

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -11,6 +11,6 @@ Namespace Plugin
Overloads Sub Add(ByVal Message As String)
Overloads Sub Add(ByVal ex As Exception, ByVal Message As String,
Optional ByVal ShowMainMsg As Boolean = False, Optional ByVal ShowErrorMsg As Boolean = False,
Optional ByVal SendInLog As Boolean = True)
Optional ByVal SendToLog As Boolean = True)
End Interface
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -6,7 +6,8 @@
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin.Attributes
<AttributeUsage(AttributeTargets.Class, AllowMultiple:=False, Inherited:=False)> Friend NotInheritable Class UseClassAsIs : Inherits Attribute
End Class
Namespace Plugin
Public Interface IPropertyProvider : Inherits IFormatProvider
Property PropertyName As String
End Interface
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -11,15 +11,12 @@ Namespace Plugin
Public UserName As String
Public SiteName As String
Public HostKey As String
Public IsChannel As Boolean
Public Options As String
Public Exists As Boolean
Public Sub New(ByVal Site As String, ByVal _Name As String)
UserName = _Name
Public Sub New(ByVal Site As String, ByVal Name As String)
UserName = Name
SiteName = Site
End Sub
Public Sub New(ByVal Site As String, ByVal _Name As String, ByVal _IsChannel As Boolean)
Me.New(Site, _Name)
IsChannel = _IsChannel
Exists = Not String.IsNullOrEmpty(Name) And Not String.IsNullOrWhiteSpace(Name)
End Sub
End Structure
End Namespace

View File

@@ -0,0 +1,36 @@
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
''' <summary>Represents errors that occur during downloading to be thrown to the root downloading function.</summary>
Public Class ExitException : Inherits Exception
''' <summary>Add only the message to the log, without adding a <see cref="StackTrace"/>. Default: <see langword="True"/>.</summary>
''' <returns><see langword="True"/> if only the message should be added to the log; otherwise the stack trace will also be added.</returns>
Public Property SimpleLogLine As Boolean = True
''' <summary>Don't add a message to the log. Default: <see langword="False"/>.</summary>
''' <returns><see langword="True"/> if the error is exit-only and there is no need to add a message to the log; otherwise add a message to the log.</returns>
Public Property Silent As Boolean = False
''' <summary>Initializes a new instance of the <see cref="ExitException"/> class.</summary>
Public Sub New()
End Sub
''' <summary>Initializes a new instance of the <see cref="ExitException"/> class with a specified error message.</summary>
''' <param name="Message">The message that describes the error.</param>
Public Sub New(ByVal Message As String)
MyBase.New(Message)
End Sub
''' <summary>
''' Initializes a new instance of the <see cref="ExitException"/> class with a specified error message
''' and a reference to the inner exception that is the cause of this exception.
''' </summary>
''' <param name="Message">The error message that explains the reason for the exception.</param>
''' <param name="InnerException">The exception that is the cause of the current exception, or a null reference (Nothing in Visual Basic) if no inner exception is specified.</param>
Public Sub New(ByVal Message As String, ByVal InnerException As Exception)
MyBase.New(Message, InnerException)
End Sub
End Class
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -7,24 +7,49 @@
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace Plugin
Public Structure PluginUserMedia
Enum Types As Integer
Undefined = 0
[Picture] = 1
[Video] = 2
[Text] = 3
VideoPre = 10
GIF = 50
m3u8 = 100
End Enum
Enum States As Integer : Unknown = 0 : Tried = 1 : Downloaded = 2 : Skipped = 3 : End Enum
Public ContentType As Integer
Public URL As String
Public MD5 As String
Public File As String
Public DownloadState As Integer
Public PostID As String
Public PostDate As Date?
Public SpecialFolder As String
Public Delegate Sub ProgressChange(ByVal Value As Double?, ByVal Maximum As Double?, ByVal Information As String)
Public Enum UserMediaTypes As Integer
Undefined = 0
Picture = 1
Video = 2
Audio = 200
Text = 4
VideoPre = 10
AudioPre = 215
GIF = 50
m3u8 = 100
End Enum
Public Enum UserMediaStates As Integer
Unknown = 0
Tried = 1
Downloaded = 2
Skipped = 3
Missing = 4
End Enum
Public Structure PluginUserMedia : Implements IUserMedia
Public Property ContentType As UserMediaTypes Implements IUserMedia.ContentType
Public Property URL As String Implements IUserMedia.URL
Public Property URL_BASE As String Implements IUserMedia.URL_BASE
Public Property MD5 As String Implements IUserMedia.MD5
Public Property File As String Implements IUserMedia.File
Public Property DownloadState As UserMediaStates Implements IUserMedia.DownloadState
Public Property PostID As String Implements IUserMedia.PostID
Public Property PostDate As Date? Implements IUserMedia.PostDate
Public Property SpecialFolder As String Implements IUserMedia.SpecialFolder
Public Property Attempts As Integer Implements IUserMedia.Attempts
Public Property [Object] As Object Implements IUserMedia.Object
End Structure
Public Interface IUserMedia
Property ContentType As UserMediaTypes
Property URL As String
Property URL_BASE As String
Property MD5 As String
Property File As String
Property DownloadState As UserMediaStates
Property PostID As String
Property PostDate As Date?
Property SpecialFolder As String
Property Attempts As Integer
Property [Object] As Object
End Interface
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -10,9 +10,9 @@ Namespace Plugin
Public Structure PropertyData
Public ReadOnly Name As String
Public ReadOnly Value As Object
Public Sub New(ByVal _Name As String, ByVal _Value As Object)
Name = _Name
Value = _Value
Public Sub New(ByVal Name As String, ByVal Value As Object)
Me.Name = Name
Me.Value = Value
End Sub
End Structure
End Namespace

View File

@@ -1,4 +1,4 @@
' Copyright (C) 2022 Andy
' Copyright (C) 2023 Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
@@ -9,8 +9,24 @@
Namespace Plugin
Public NotInheritable Class PropertyValue : Implements IPropertyValue
Public Event ValueChanged As IPropertyValue.ValueChangedEventHandler Implements IPropertyValue.ValueChanged
Public Event CheckedChanged As IPropertyValue.ValueChangedEventHandler
Public Property [Type] As Type Implements IPropertyValue.Type
Public Property OnChangeFunction As IPropertyValue.ValueChangedEventHandler
Public Property OnCheckboxCheckedChange As EventHandler(Of PropertyValueEventArgs)
Private _Checked As Boolean = False
Public Property Checked As Boolean
Get
Return _Checked
End Get
Set(ByVal IsChecked As Boolean)
_Checked = IsChecked
If Not _Initialization Then
If Not OnCheckboxCheckedChange Is Nothing Then OnCheckboxCheckedChange.Invoke(Me, EventArgs.Empty)
RaiseEvent CheckedChanged(_Checked)
End If
End Set
End Property
Private _Initialization As Boolean = False
''' <inheritdoc cref="PropertyValue.New(Object, Type, ByRef IPropertyValue.ValueChangedEventHandler)"/>
''' <exception cref="ArgumentNullException"></exception>
Public Sub New(ByVal InitValue As Object)
@@ -41,10 +57,26 @@ Namespace Plugin
End Get
Set(ByVal NewValue As Object)
_Value = NewValue
If Not OnChangeFunction Is Nothing Then OnChangeFunction.Invoke(Value)
RaiseEvent ValueChanged(_Value)
If Not _Initialization Then
If Not OnChangeFunction Is Nothing Then OnChangeFunction.Invoke(Value)
RaiseEvent ValueChanged(_Value)
End If
End Set
End Property
Public Sub BeginInit()
_Initialization = True
End Sub
Public Sub EndInit()
_Initialization = False
End Sub
Public Sub Clone(ByVal Source As PropertyValue)
_Initialization = True
Type = Source.Type
OnChangeFunction = Source.OnChangeFunction
_Value = Source._Value
_Checked = Source._Checked
_Initialization = False
End Sub
End Class
Public Interface IPropertyValue
''' <summary>Event for internal exchange</summary>
@@ -55,4 +87,8 @@ Namespace Plugin
''' <summary>Property value</summary>
Property Value As Object
End Interface
Public Class PropertyValueEventArgs : Inherits EventArgs
Public Property Checked As Boolean = False
Public Property ControlEnabled As Boolean = True
End Class
End Namespace

View File

@@ -102,9 +102,12 @@
</ItemGroup>
<ItemGroup>
<Compile Include="Attributes\Attributes.vb" />
<Compile Include="Interfaces\IDownloadableMedia.vb" />
<Compile Include="ObjectInterfaces\IPropertyProvider.vb" />
<Compile Include="Objects\ExchangeOptions.vb" />
<Compile Include="ObjectInterfaces\ILogProvider.vb" />
<Compile Include="Interfaces\IPluginContentProvider.vb" />
<Compile Include="Objects\ExitException.vb" />
<Compile Include="Objects\PluginUserMedia.vb" />
<Compile Include="Interfaces\ISiteSettings.vb" />
<Compile Include="ObjectInterfaces\IThrower.vb" />

View File

@@ -0,0 +1,3 @@
[*.vb]
# Modifier preferences
file_header_template = Copyright (C) Andy https://github.com/AAndyProgram\nThis program is free software: you can redistribute it and/or modify\nit under the terms of the GNU General Public License as published by\nthe Free Software Foundation, either version 3 of the License, or\n(at your option) any later version.\n\nThis program is distributed in the hope that it will be useful,\nbut WITHOUT ANY WARRANTY; without even the implied warranty of\nMERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the\nGNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License\nalong with this program. If not, see <https://www.gnu.org/licenses/>

View File

@@ -0,0 +1,33 @@
' Copyright (C) Andy https://github.com/AAndyProgram
' This program is free software: you can redistribute it and/or modify
' it under the terms of the GNU General Public License as published by
' the Free Software Foundation, either version 3 of the License, or
' (at your option) any later version.
'
' This program is distributed in the hope that it will be useful,
' but WITHOUT ANY WARRANTY
Namespace [Shared]
Public Module Functions
Public Const NewReleaseFolderName As String = "__NewRelease"
Public Function GetCurrentMaxVer(Optional ByVal Path As SFile = Nothing) As Version
Try
If Path.IsEmptyString Then Path = Application.StartupPath.CSFileP
If Path.Exists(SFO.Path, False) Then
Dim versions As New List(Of Version)
Dim v As FileVersionInfo
With SFile.GetFiles(Path, "*.exe",, EDP.ReturnValue).ListIfNothing.Where(Function(f) f.Name = "SCrawler" Or f.Name = "YouTubeDownloader")
If .ListExists Then
For Each f As SFile In .Self
v = FileVersionInfo.GetVersionInfo(f)
versions.Add(New Version(v.ProductVersion))
Next
End If
End With
If versions.Count > 0 Then Return versions.Max
End If
Catch
End Try
Return Nothing
End Function
End Module
End Namespace

View File

@@ -0,0 +1,13 @@
'------------------------------------------------------------------------------
' <auto-generated>
' This code was generated by a tool.
' Runtime Version:4.0.30319.42000
'
' Changes to this file may cause incorrect behavior and will be lost if
' the code is regenerated.
' </auto-generated>
'------------------------------------------------------------------------------
Option Strict On
Option Explicit On

Some files were not shown because too many files have changed in this diff Show More