github/supysonic
1
0
Fork 0
mirror of https://github.com/spl0k/supysonic.git synced 2024-12-23 01:16:18 +00:00
Code Issues Releases Wiki Activity
670 Commits 6 Branches 18 Tags 2.6 MiB
dcef74ca70
Commit Graph

328 Commits

Author SHA1 Message Date
spl0k
7f8369cac4 Splitting daemon stuff 2019-04-22 15:43:29 +02:00
spl0k
9850cae241 CLI can now scan in background 
Some issues:
- can't queue scans, not really user-friendly
- scanning seems to touch files which are in turn picked by the watcher
 2019-04-20 17:10:55 +02:00
spl0k
75540bb376 Daemon can now scan on its own 
Scanning threads cannot be stopped
 2019-04-20 17:03:19 +02:00
spl0k
22a207c79f Nicify commands sent to the daemon 2019-04-20 11:51:34 +02:00
spl0k
3924ada03e Added a config variable to define if the watcher should be started 2019-04-13 17:14:03 +02:00
spl0k
c7f1499d46 Make Python2 happy 2019-04-13 16:02:32 +02:00
spl0k
57aea188c2 Fixed potential error/freeze when getting a secret key 2019-04-10 21:21:25 +02:00
spl0k
afbd84a5b9 Changed how the DaemonClient get its address + better exception 
I'm still not satisfied but can't come up with a good solution
 2019-04-10 21:15:18 +02:00
spl0k
751f00dac8 Daemon now listen for remot commands, adding/removing watched folders on the fly 
#77 part 5
 2019-04-08 19:16:41 +02:00
spl0k
970ee6ee3c Merge branch 'master' into daemon-rework 2019-04-07 16:44:17 +02:00
spl0k
c938f225e9 Store cookie key in db rather than cache 2019-04-07 16:35:26 +02:00
spl0k
92fed40f87 Use mtime as folder/track creation date 
Closes #145
 2019-04-07 15:25:13 +02:00
spl0k
db2799ef7e Reworked how the watcher is started 2019-03-31 18:06:45 +02:00
spl0k
270fa9883b Removing executable flag from files that shouldn't have it 2019-03-31 16:37:56 +02:00
spl0k
7ff56bc4ad Add indexes on all foreign keys 
Should help with query performance issues such as #140
 2019-03-24 17:26:14 +01:00
spl0k
780c96c404 Fixed issues with float bitrate and empty tags 
Thanks to @nextfullstorm
Closes #139
 2019-03-23 17:08:46 +01:00
spl0k
e2cc51a0f0 Fixed issues when the watcher is set to log to /dev/null 2019-03-23 16:20:19 +01:00
spl0k
c0d9d616c8 Truncate artist, album and title when scanning 
Closes #138
 2019-03-10 11:52:18 +01:00
spl0k
a73d42b9d8 Make MySQL happy with binary data 
Closes #137
 2019-03-10 11:38:55 +01:00
spl0k
5367f068e1 Giving credit where credit is due 2019-02-09 16:19:30 +01:00
spl0k
cf846e88ee Merge remote-tracking branch 'pR0Ps/feature/transcode-cache' 2019-02-09 15:49:30 +01:00
spl0k
29429bbbf2 Sanitize tags a bit when readong them 
Closes #133, closes #134
 2019-02-09 15:48:16 +01:00
Carey Metcalfe
07342529e0 Ensure transcoding failures are never cached 
By re-raising any exceptions that `transcode` encounters while running
the transcode, it ensures that `set_generated` will also see the
exception, preventing it from caching an incomplete transcode.
 2019-02-03 23:24:26 -05:00
Carey Metcalfe
0ac2376e07 Fix issues with hanging transcoding processes 
When a connection that is consuming a generated response is closed,
Flask closes the generator making it raise the special `GeneratorExit`
exception when the program tries to yield from it again. Because the
`transcode` function was called (returning a generator) before being
passed into `set_generated`, the exception was being handled in the
wrong order.

By passing the `transcode` function to `set_generated` and letting
`set_transcode` call it to return a generator while generating the
response for the client, the exception properly bubbles up through
`transcode` into `set_generated`. This allows both of them to handle it
properly by stopping the subproceses and not caching the incomplete
response data respectively.
 2019-02-03 23:23:00 -05:00
Carey Metcalfe
f106923f17 Remove unused 'Cache.protect' method 2019-02-03 22:56:03 -05:00
spl0k
53d8660683 Don't pretty print XML output 2019-02-03 19:35:05 +01:00
Carey Metcalfe
7fa1501112 Cache lyrics obtained from ChartLyrics 
Lyrics are stored as json in a zlib-compressed file.
 2019-01-19 15:39:10 -05:00
Carey Metcalfe
3db489aa7b Add compatibility shims for Python 2.7 
- `os.scandir` (provided by a 3rd party package in 2.7)
- `os.replace` (doesn't exist in 2.7 - have to use `os.rename` instead)
- `os.utime` (the times param is required in 2.7)
 2019-01-19 15:37:09 -05:00
Carey Metcalfe
65d49a04c9 Implement a cache manager for album art and transcodes 
Quick summary
-------------
- Adds a Cache class (plus tests for it) that provides an API for
  managing a cache of files on disk
- Adds two new settings to the configuration file: `cache_size` (default
  512MB) and `transcode_cache_size` (default 1GB).
- Creates two cache managers using the settings above: one for general
  stuff (currently album art) and one for transcodes
- Adds the caching of transcoded files to disk for future use
- Modifies the existing image caching to use the cache manager

Longer explanations and justifications
--------------------------------------
The reason I separated out transcodes into an entirely separate cache is
that I could imagine a single transcode pushing out a ton of smaller
images or other cached content. By separating them it should reduce the
number of deletes caused by adding something to the cache.

The cache manager allows for caching a value from a generator via
passthrough. This means that a generator can be transparently wrapped to
save its output in the cache. The bytes from the generator will be
written to a temp file in the cache and yielded back. When it completes,
the temp file will be renamed according to the provided cache key. This
is how caching transcoded music is implemented.

If multiple generators for the same key are started, they will all write
to individual temp files until they complete and race to overwrite each
other. Since the key should uniquely represent the content it indexes
the files will be identical so overwriting them is harmless.

The cache will store everything for a minimum amount of time
(configurable, default is set at 5 minutes). After this time has
elapsed, the data can be deleted to free up space. This minimum is so
that when you cache a file to the disk you can expect it to be there
after, even if another large file is added to the cache and requests
that some files are deleted to make space.

To ensure that a file will not be paged out of the cache regardless of
the minimum time, there is a `protect` context manager that will refuse
the delete the key from the cache as long as it's active.

The cache has a maximum size, however this is more of a recommendation
as opposed to a hard limit. The actual size will frequently exceed the
limit temporarily until something can be paged out.
 2019-01-19 15:37:08 -05:00
spl0k
3b6186ebfc JSON: '_value_' -> 'value' 
Closes #136
 2019-01-19 15:04:56 +01:00
Carey Metcalfe
69c43b749b Use HTTPS when communicating with Last.fm 2019-01-17 04:48:35 -05:00
spl0k
25c5c8277e Add support for downloadong folders and albums 
Ref #131
 2019-01-13 17:18:27 +01:00
spl0k
6b86f3a43a Added getSongsByGenre 2018-12-08 17:42:20 +01:00
spl0k
ac306f2725 Added method to list genres 2018-12-08 17:18:58 +01:00
spl0k
b195aca12f watcher: log to stderr when no logfile is set 2018-11-11 19:29:58 +01:00
spl0k
f5bab00a1c Small touches on logging 
Most notably not relying in Flask's logger, as this cause issues
at the end of tests on Python 2.7
Fixes #96
 2018-11-11 18:58:46 +01:00
spl0k
8132e38d46 Delete track dependents before tracks on folder deletion 
Fixes #129
 2018-11-01 14:02:12 +01:00
Taizo Simpson
306fc202bd
Add a timeout when fetching remote resources 2018-10-31 13:20:38 -04:00
spl0k
a187a73122 Commit right after registering a new client 
Closes #117
 2018-10-28 15:14:48 +01:00
spl0k
fd5d53f253 Removed useless transaction handling from latest migrations 2018-10-20 16:03:46 +02:00
spl0k
889b83b761 Merge remote-tracking branch 'TaizoSimpson/master' 2018-10-20 15:59:38 +02:00
Taizo Simpson
25cb354f78
Merge branch 'embed_as_metadata' 2018-10-14 15:18:16 -04:00
Taizo Simpson
f7dc6292fd
Switched from python to sql migration 2018-10-14 15:09:06 -04:00
spl0k
ed0a8fccd0 Merge remote-tracking branch 'pR0Ps/bugfix/kill-transcode-processes' 2018-10-13 14:26:05 +02:00
spl0k
bfa13ce3c9 Fixed meta table creation on mySQL 
Fixes #121
 2018-10-13 14:20:44 +02:00
Taizo Simpson
c89395b220
Annotated method as static so python2 stops complaining 2018-10-12 19:36:21 -04:00
Taizo Simpson
753b4d9df8
Updated schema version 2018-10-12 19:13:12 -04:00
Taizo Simpson
066658f167
Consider embeded art when serializing albums, add relevant test 2018-10-12 19:07:48 -04:00
Taizo Simpson
98cfe19a38
Removed unnecessary variable initialization 2018-10-12 18:33:04 -04:00
Taizo Simpson
6efb8e8c2d
Added some migrations 
WARNING: Still not ready for production — schema version needs incrementing, and testing still required
 2018-10-12 13:53:56 -04:00
Carey Metcalfe
9ac30419ab Kill transcoding processes on error 
Asking nicely with a SIGTERM doesn't cause the transcoding process(es)
to exit. Using SIGKILL gets the job done.

This was verified by manually sending SIGTERM and SIGKILL signals to
hung transcoding processes, as well as getting a client to abort stream
requests before they had completed.

Fixes #55
 2018-10-12 02:38:09 -04:00
Taizo Simpson
2c4ec6c0e8
Whether or not a track has embeded art is tracked just like any other metadata, to reduce disk r/w and mem usage and a lot of other things. 
WARNING: This needs migrations before being merged to master
 2018-10-10 12:59:20 -04:00
Taizo Simpson
74fba8840e
Fixed error when shrinking a cached embeded cover 2018-10-09 16:37:00 -04:00
Taizo Simpson
00b043dda6
Used Python2-friendly syntax in creating new directories 2018-10-08 21:28:24 -04:00
Taizo Simpson
4bbcbd0acf
Art extraction is now done as a method of Track; cover_art() only extracts cover art if given a track ID; Cache embeded art 2018-10-08 20:05:45 -04:00
Taizo Simpson
7106d95cee
Added support for ID3 based metadata, OGG, and FLAC in album art extraction 2018-10-08 16:34:44 -04:00
Taizo Simpson
befc460120
Added support for embeded album art in MP3s 2018-10-08 15:31:15 -04:00
Carey Metcalfe
396df4f1ca Stringify Exceptions when creating SubsonicAPIExceptions from them 
This prevents a `TypeError` from being raised when the
`SubsonicAPIException` is serialized to JSON in preparation for being
returned to the client.
 2018-10-07 16:17:26 -04:00
Michael Sabin
8c38292abd Expand path when adding a folder 
Expand the path using the system environment
before generating the absolute path.
For example, ~ was not being expanded to
the user's home directory.
 2018-09-29 22:53:10 -07:00
spl0k
5a065c75ae Merge branch 'master' into db-setup 2018-09-15 16:04:43 +02:00
spl0k
59f7fd62b0 Improved transcoding description and added client-side validation 
Closes #115
 2018-09-15 16:02:36 +02:00
spl0k
1a15b95155 Merge branch 'master' into issue90 2018-09-08 15:37:08 +02:00
spl0k
670bef6a91 Added support for python migrations 2018-09-02 17:26:18 +02:00
spl0k
659e46611a Fixed tests 2018-08-29 18:18:31 +02:00
spl0k
e84459d627 First throw at automatic database migrations 
Only supporting SQL scripts for now
 2018-08-29 17:46:59 +02:00
spl0k
2568b9bc91 Added a small table to store the schema version 
Defined in a dedicated 'pony database', allowing to check only this table
to determine if we need to create the tables, and so existing tables getting
a new attribute won't trigger a table creation
 2018-08-29 16:09:41 +02:00
spl0k
dbf817ea9e Automatically create DB tables if needed 2018-08-29 14:56:28 +02:00
spl0k
9c58b695ff Watcher now handles cover art 
Closes #92
 2018-08-28 17:44:32 +02:00
spl0k
9736622ce1 Recently played API returns albums/folders that have been played at least once 
Fixes #107
 2018-08-27 15:51:01 +02:00
spl0k
38f718b4df Using new pony on_connect decorator 
Revert "Restore SQLite case insensitive LIKE"
This reverts commit a08689e8c0.
 2018-08-11 16:16:34 +02:00
spl0k
a08689e8c0 Restore SQLite case insensitive LIKE 
See ponyorm/pony#367
 2018-08-05 15:35:29 +02:00
spl0k
dd33e8da14 Fixed LastFM account linking 2018-08-04 13:12:20 +02:00
spl0k
1b0cc54581 Workaround new pony QueryResult behaviour 
Ref ponyorm/pony#369
 2018-08-04 12:24:29 +02:00
spl0k
b25e943e4e Removed bulk deletes from scanner 
Was causing either cache issues or constraint errors
Fixes #103, #102
 2018-07-29 17:58:20 +02:00
spl0k
78cf84e136 Remove invalid folders before scanning for covers 
Fixes #101
 2018-06-24 13:19:04 +02:00
spl0k
b43a70a045 Cover art discovery: handle badly encoded files 2018-05-21 16:37:21 +02:00
spl0k
405a26a20a Don't restrict cover art to 'cover.jpg' files 
Ref #81
 2018-05-21 16:16:06 +02:00
spl0k
918cd11262 Allow missing/empty tags 
Closes #94, closes #99
 2018-05-20 17:35:32 +02:00
spl0k
a7cdeb0b2d Don't bulk delete folders 
Fixes issue with MariaDB<10.3.1
Ref #99
 2018-05-20 13:38:17 +02:00
spl0k
2b1977892e Generate a secret key only once 2018-04-01 12:32:36 +02:00
hhm
6bb98085bc B"H allow accessing public playlists even when not admin 2018-03-21 23:11:28 -04:00
spl0k
6ac969e2c5 Index folder and track path 
Speeds up scanning
 2018-03-17 23:25:26 +01:00
spl0k
b72daea109 Improved folder deletion 2018-03-15 20:50:01 +01:00
spl0k
b4a9a21671 Don't distinct random queries 
Fixes #89
 2018-03-12 21:06:57 +01:00
spl0k
e5d9050fa7 Improved transcoding configuration parsing 2018-03-11 21:29:17 +01:00
spl0k
9eb1bf63d7 API: stream.view: added support for 'estimateContentLength' 2018-03-10 23:04:57 +01:00
spl0k
a6b894c586 API: marked explicitly unsupported methods/parameters as such 2018-03-10 22:15:40 +01:00
spl0k
adb4e7e89b API: improved client preference passing 2018-03-10 18:35:00 +01:00
spl0k
0988d4319e Unified file headers 2018-03-04 21:49:56 +01:00
spl0k
4cbc8765e0 Refactored FolderManager to raise exceptions 
rather than returning status codes
 2018-03-04 17:36:35 +01:00
spl0k
ef9e7af026 Refactored UserManager to raise exceptions 
rather than returning status codes
 2018-03-02 22:51:49 +01:00
spl0k
8bf488fab2 Refactored star/unstar error handling 2018-02-28 20:12:34 +01:00
spl0k
5188976e6f Merge branch 'master' 2018-02-25 22:36:39 +01:00
spl0k
58b8a1171a More error refactoring 2018-02-25 22:28:47 +01:00
spl0k
177b0cce0d Refactoring API error handling 2018-02-25 11:39:26 +01:00
spl0k
ba61281ba1 Allow extra db connection args. Set utf8mb4 as the default charset for MySQL 2018-02-21 22:13:35 +01:00
spl0k
43b197a95e Wrapping all request handling in a database transaction 2018-02-14 22:48:44 +01:00
spl0k
e3ccf0809f Bumping the version, just because 2018-02-14 19:53:17 +01:00
spl0k
005ae4803b More formatter refactoring 2018-02-11 12:40:10 +01:00
spl0k
27b9c232c2 Small refactoring of supysonic.api.get_entity 2018-01-30 21:56:06 +01:00
spl0k
66b6eb341e Tests: removed context/module unload dance 
Was a workaround for issues fixed by blueprints
 2018-01-29 22:09:28 +01:00
spl0k
7455711b60 Frontend as blueprint 
Ref #76
 2018-01-29 21:37:19 +01:00
spl0k
0de87e64b0 API as blueprint 
Ref #76
 2018-01-28 22:50:21 +01:00
spl0k
aedda4f642 Add CORS header on JSON responses only 2018-01-27 17:18:30 +01:00
spl0k
b33e8ae6d1 Set mimetype when creating the response, don't try to fix it afterwards 
Ref #76
 2018-01-27 15:18:44 +01:00
spl0k
8275966db0 Dropped simplejson dependency 
Ref #79
 2018-01-21 22:02:32 +01:00
spl0k
c6992a4726 Fixed reading malformed tags 
Fixes #86
 2018-01-21 14:42:27 +01:00
spl0k
2df4de47a8 Replaced most of catch-all exception handlers 
Closes #82
 2018-01-20 22:46:41 +01:00
spl0k
3fd748cc5a Scanner: handle permission errors 2018-01-20 21:44:20 +01:00
spl0k
954c75bc35 Ignore files with a badly encoded path 
Closes #85
 2018-01-20 18:26:58 +01:00
spl0k
bfe6da12d7 Dropped 'future' dependency 2018-01-18 21:10:01 +01:00
spl0k
fd67eb9cd4 Fixed DeprecationWarnings 2018-01-16 23:16:13 +01:00
spl0k
420003dfc1 Py3: fixed remaining errors (hidden due to skipped tests) 
Ref #75
 2018-01-16 22:19:24 +01:00
spl0k
c36d1813ee Fixed setting client preferences 2018-01-16 19:45:28 +01:00
spl0k
7edb246b1e Py3: str/bytes, iterators, etc. 
It seems to work on Python 3 now!
Ref #75
 2018-01-11 23:08:53 +01:00
spl0k
1a79fe3d70 Py3: basestring 
Ref #75
 2018-01-08 23:13:50 +01:00
spl0k
8674965c03 Read config files without interpolation 
Fixes #84
 2018-01-08 19:17:00 +01:00
spl0k
1605fcd202 Py3: imports, exceptions, dicts 
Ref #75
 2018-01-06 00:33:45 +01:00
spl0k
d807f26ccb Provided migration scripts for existing storm databases 2017-12-30 19:06:43 +01:00
spl0k
df63919634 Look at my pony, my pony is amazing 2017-12-23 22:59:04 +01:00
spl0k
53fd4865cb Get on my pony 2017-12-21 23:29:00 +01:00
spl0k
6daedc6919 I'm on a pony 2017-12-19 23:16:55 +01:00
spl0k
2428ffeb57 Scanner, CLI and watcher are on a pony 2017-12-17 23:25:34 +01:00
spl0k
a4b9a97271 All managers get a pony 2017-12-16 17:42:53 +01:00
spl0k
6bd61e0388 Getting out of the storm on a pony 2017-12-16 13:53:43 +01:00
spl0k
8046457661 Relative imports 2017-12-12 22:24:18 +01:00
spl0k
0a47ef89a5 CLI: fixed foreign key errors on user deletion 2017-12-10 15:11:50 +01:00
spl0k
d3271e7174 CLI: added ability to set stdin/stdout/stderr 2017-12-10 14:31:04 +01:00
spl0k
5236abf6ff Moved CLI code to package 2017-12-08 22:02:12 +01:00
spl0k
bb72ce8f7c Slight changes to watcher 
Configurable scan delay
Test friendliness
 2017-12-05 22:48:55 +01:00
spl0k
c515a25b69 Config sample update 2017-11-29 22:55:39 +01:00
spl0k
9ffe40aa2c Reactivated config-dependent test 2017-11-27 22:46:09 +01:00
spl0k
a62976ba9d Reworked config handling 2017-11-27 22:30:13 +01:00
spl0k
510d9e6dc7 Added some validation to scanner 2017-11-25 22:03:42 +01:00
spl0k
18383a6cc4 Removed users import/export 
That was some method I was using by the time I deleted/recreated the
database on each schema change
 2017-11-24 19:24:00 +01:00
spl0k
c947219f59 Web UI user section: validation 2017-11-24 19:19:24 +01:00
spl0k
0353a8a1bc Web UI user section: factoring some usual validation with a decorator 2017-11-22 23:11:58 +01:00
spl0k
b7e9914246 Web UI: split GET and POST handlers 2017-11-22 22:39:54 +01:00
spl0k
b998bb0684 Web UI: reworked how data is passed to templates 
Logged user is pushed in request data
No more 'username' session key
Small fixes along the way
 2017-11-22 22:07:06 +01:00
spl0k
1488f29e9f Fixed playlist related errors 2017-11-20 21:31:10 +01:00
spl0k
aac5ee6936 Small None-formatting / mimetype fix (and (c) dates) 2017-11-17 19:46:35 +01:00
spl0k
da857b9ceb Reworked star/unstar error handling 
Don't stop at the first error anymore, instead treat all provided ids
Potential errors are aggregated, if there's only one it is returned
as-is, if there are more they are all nested in a top-level error whose
code is 0 if there are different error codes, or their value if all
error codes are the same.
This error-nesting doesn't validate against the XSD, but the Subsonic
API gives absolutely no information on how errors should be handled.
And reverse-engineering is not in my line of work
 2017-11-14 23:16:58 +01:00
spl0k
c709fa7e29 Better errors on listings 2017-11-11 00:12:10 +01:00
spl0k
336f556ced Improved playlists error handling 2017-11-08 23:18:41 +01:00
spl0k
0602bed2e8 Fixed error in old search and handled newerThan parameter 2017-11-03 23:14:39 +01:00
spl0k
b20c151765 Prevent server error when browsing on tracks that have been deleted 2017-11-03 23:12:38 +01:00
spl0k
d8c3b9fa88 User management error code tweaks 2017-11-01 20:52:59 +01:00
spl0k
7de57cb680 Small API content-type and error codes adjustments 2017-10-29 16:00:39 +01:00
spl0k
8f5fcc82f5 Response formatting tests 2017-10-28 21:18:34 +02:00
spl0k
033a86381b Removed password decoding from UserManager 
Decode only when passwords are coming from API query parameters
 2017-10-28 12:41:34 +02:00