2023.8.6.0

Plugins.Attributes: add 'DependentFields' attribute
Plugins.IPluginContentProvider: add 'Options' and 'IsSubscription' properties
Plugins.ISiteSettings: add 'SubscriptionsAllowed' property
Plugins.ExchangeOptions: add 'Options' field
Plugins.Attributes.PropertyUpdater: replace 'Dependencies' with 'Arguments'

YT: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties; add the ability to store download locations; add 'DownloadLocation' and 'DownloadLocationsCollection' objects
YT.IDownloaderSettings: add 'OutputPathAskForName' and 'OutputPathAutoAddPaths' properties
YT.Downloader: fixed bug with re-saving elements when loading a video list; fixed bug when files were not deleted when clicking on the delete button; fixed a bug that caused the video to redownload; download job removes elements at wrong indexes; added skipping of downloaded elements in the job; fixed a bug, pending option did not change after download complete
YT.YouTubeMediaContainerBase: add '_MediaStateOnLoad' field and 'NeedToSave' function; update the 'Save' function to prevent saving a file when a download is complete and the file has already been saved; update code for new yt-dlp version

Fixed cache deletion errors
Add user queue
Add global locations
API.Base.SiteSettingsBase: implement 'SubscriptionsAllowed' property; remove request headers with null values on save; add '_AllowUserAgentUpdate' parameter
API.Base.Structures: add 'SiteModes' enum
API.Base.UserDataBase: add 'Erase' button; implement 'Options' and 'IsSubscription' properties; add 'SpecialLabels' property; update 'LVIKey'; update 'FitToAddParams' function; add 'EraseData' function; user colors; Not UserExists notification, UserQueue support
API.Base: add 'DeclaredNames'
API.Instagram: remove default values for headers; disable updating UserAgent from global; check for a new username for non-existent users
API.Mastodon: bypass new inherited twitter options; update names and headers
API.OnlyFans: make 'HH_BROWSER' property nullable; remove 'HH_BROWSER' from required; fix username bug (dots); handling of 504 and 429 errors; add 'DownloadHighlights' and 'DownloadChatMedia' options; add 'UserExchangeOptions'; fixed incorrect error handler
API.PathPlugin: fixed incorrect detection of path existence
API.Pinterest: add 'SpecialLabels'
API.PornHub: add new video regex; remove old regex; added 'DownloadUploaded', 'DownloadTagged', 'DownloadPrivate' and 'DownloadFavorite' properties to 'SiteSettings', 'UserData' and 'UserExchangeOptions'; update regex to define user; added downloading search queries; update 'GetUserUrl' function; hide unnecessary 'RegexFieldsTextBecameNullException' errors; add subscriptions
API.Reddit: add 'SpecialLabels'; add bearer token and its refresh interval; add OAuth; add additional options
API.RedGifs: add 'DependentFields' for 'Token'
API.ThisVid: add 'DownloadFavourite' option; add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; updating cookies issue
API.TikTok: rewrite algorithms
API.Twitter: add 'UseAppropriateModel', 'UseNewEndPointSearch', 'UseNewEndPointProfiles', 'AbortOnLimit', 'DownloadAlreadyParsed', 'MediaModelAllowNonUserTweets' properties; remove old commented code; remove 'TwitterPic_400' and replace with 'TwitterIcon_32.ToBitmap'; add 'DownloadModelForceApply' user option; update environment to GDL 1.25.8; fixed gifs downloading; fix typo in 'ReparseMissing'; update names
API.UserDataBind: prevent adding site-specific labels when adding to a collection
API.Xhamster: add downloading search queries, tags, categories; add 'SpecialLabels'; add additional nodes for channels; add subscriptions
API.XVIDEOS: add downloading search queries, tags, categories; add 'SpecialLabels'; add subscriptions; changed users creation method; add subscriptions
API.YouTube: add subscriptions
AutoDownloader: add new group subscription options; update predicates; fixed excluded labels and sites in default mode; update notifications; add an additional skip options, add 'Force start' option
DownloadedInfoForm: add subscriptions; fixed size/location bug; hide unnecessary error (refill)
Feed: add subscriptions; update filters; add 'Ctrl+G' shortcut
FeedMedia: add subscriptions; fixed 'webm' bug; add title for subscription media; add site icon to post; user colors; always using 'FriendlyName' instead of 'UserName' if it exists
DownloadGroup, GroupDefaults, GroupParameters: add subscription and 'UsersCount' options
MissingPostsForm: add 'BTT_DELETE_ALL'
VideoDownloaderForm, DownloaderUrlForm, DownloaderUrlsArrForm: add download locations support
VideoDownloaderForm: add subscriptions support
GlobalSettingsForm: add new properties
UserCreatorForm: add subscriptions; add 'Options' support (of 'ExchangeOptions'); user colors
ListImagesLoader: add subscription colors; user colors
MainFrame: add subscriptions; add filters by subscription and user; update predicates
NuGet: update 'LibVLCSharp', 'LibVLCSharp.WinForms', 'VideoLAN.LibVLC.Windows'
DownloadableMediaHost: update 'Save' function
PropertyValueHost: fix 'CaptionWidth' bug; add 'Dependents'
SettingsHost: add 'Dependents'
UserDataHost: add 'Options' and 'IsSubscription' properties
SettingsCLS: implement new 'IDownloaderSettings' properties; add 'CacheSnapshots'; add 'DownloadLocations'; add new properties
UserInfo, UserFinder: add subscriptions
UserSearchForm: fixed search by name bug
This commit is contained in:
Andy
2023-08-06 18:16:07 +03:00
parent bade8666d5
commit df06a86651
179 changed files with 9145 additions and 2041 deletions

View File

@@ -9,9 +9,7 @@
Imports System.Runtime.CompilerServices
Imports PersonalUtilities.Functions.XML.Objects
Imports PersonalUtilities.Functions.RegularExpressions
Imports PersonalUtilities.Forms.Toolbars
Imports PersonalUtilities.Tools
Imports PersonalUtilities.Tools.Web
Imports SCrawler.API.Base
Imports SCrawler.Plugin.Hosts
Imports SCrawler.DownloadObjects
@@ -21,6 +19,8 @@ Friend Module MainMod
Friend ReadOnly LinkPattern As RParams = RParams.DMS("[htps:]{0,6}[/]{0,2}(.+)", 1)
Friend ReadOnly FilesPattern As RParams = RParams.DM("[^\./]+?\.\w+", 1, EDP.ReturnValue)
Friend Delegate Sub NotificationEventHandler(ByVal Sender As SettingsCLS.NotificationObjects, ByVal Message As String)
Friend Delegate Sub UserDownloadStateChangedEventHandler(ByVal User As IUserData, ByVal IsDownloading As Boolean)
Friend Delegate Function PathMoverHandler(ByVal User As UserInfo, ByVal DestinationPattern As SFile) As SFile
Friend Const LVI_TempOption As String = "Temp"
Friend Const LVI_FavOption As String = "Favorite"
Friend Const LVI_CollectionOption As String = "Collection"
@@ -68,6 +68,14 @@ Friend Module MainMod
[Default] = 0
Virtual = 1
End Enum
Friend Enum PathCreationModel As Integer
Undefined = -1
Path = 1
Path_UserName = 2
Path_UserSite_UserName = 3
DefaultUser = Path_UserSite_UserName
Collection = 4
End Enum
Friend Downloader As TDownloader
Friend InfoForm As DownloadedInfoForm
Friend VideoDownloader As STDownloader.VideoDownloaderForm
@@ -77,6 +85,8 @@ Friend Module MainMod
Friend ReadOnly DateTimeDefaultProvider As New ADateTime(ADateTime.Formats.BaseDateTime)
Friend ReadOnly FeedVideoLengthProvider As New ADateTime("hh\:mm\:ss") With {.TimeParseMode = ADateTime.TimeModes.TimeSpan}
Friend ReadOnly UserExistsPredicate As New FPredicate(Of IUserData)(Function(u) u.Exists)
Friend ReadOnly UserExistsSubscriptionsPredicate As New FPredicate(Of IUserData)(Function(u) u.Exists And u.IsSubscription)
Friend ReadOnly UserExistsNonSubscriptionsPredicate As New FPredicate(Of IUserData)(Function(u) u.Exists And Not u.IsSubscription)
Friend ReadOnly LogConnector As New LogHost
Friend DefaultUserAgent As String = String.Empty
#Region "File name operations"
@@ -91,6 +101,11 @@ Friend Module MainMod
End Class
#End Region
Friend Property MainProgress As MyProgressExt
Friend Sub ShowOperationCanceledMsg(Optional ByVal MsgTitle As String = Nothing)
Dim m As New MMessage("Operation canceled")
If Not MsgTitle.IsEmptyString Then m.Title = MsgTitle
m.Show()
End Sub
Friend Function GetLviGroupName(ByVal Host As SettingsHost, ByVal IsCollection As Boolean) As ListViewGroup()
Dim l As New List(Of ListViewGroup)
Dim t$
@@ -125,8 +140,11 @@ Friend Module MainMod
End If
End Function
Friend Sub CheckVersion(ByVal Force As Boolean)
If Settings.CheckUpdatesAtStart Or Force Then _
GitHub.DefaultVersionChecker(My.Application.Info.Version, "AAndyProgram", "SCrawler",
Settings.LatestVersion.Value, Settings.ShowNewVersionNotification.Value, Force)
With Settings
If .CheckUpdatesAtStart Or Force Then
ShowProgramInfo(.ProgramText.Value.IfNullOrEmpty("SCrawler"), My.Application.Info.Version, True, Force, .Self, False,
.LatestVersion.Value, .ShowNewVersionNotification.Value, .ProgramDescription)
End If
End With
End Sub
End Module