wbrawner/server
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
49707 commits 157 branches 562 tags 714 MiB
Commit graph

98 commits

Author SHA1 Message Date
Robin Appelman
fcd637b938 dont delete cache entries if deleting an object from object store failed 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-09-26 14:14:57 +00:00
Marcel Klehr
21822512dc Fix File#putContents(string) on ObjectStorage 
Signed-off-by: Marcel Klehr <mklehr@gmx.net>
 2019-07-17 20:39:24 +00:00
Roeland Jago Douma
b62b835f6c Use HTTP1.1 to read S3 objects 
Some of the READs otherwise use HTTP/1.0 which is not always supported
by all backends. HTTP/1.1 is there since 1999 way longer than S3 so safe
to assume it is always there IMO.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-07-10 13:20:38 +00:00
Roeland Jago Douma
b6fe0f1f9a Always use a temp file for swift writes 
Apparently the if statement doesn't work in all cases (even if I could
not reproduce it). So for the time being we will just not directly
stream to swift.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-04-24 07:01:36 +00:00
Roeland Jago Douma
1b27e9578d
Merge pull request #14424 from nextcloud/fix/13554/swift_to_tmp 
Use a tmp file for swift writes
 2019-03-06 15:33:15 +01:00
Morris Jobke
debd32b461
Merge pull request #14175 from skydiablo/master 
Fix ObjectStorage expired token (v3)
 2019-03-05 18:59:45 +01:00
Roeland Jago Douma
1153123b90
Use a tmp file for swift writes 
Else this leads to a seekable stream error with chunked uploads

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-03-04 13:11:59 +01:00
Roeland Jago Douma
df55781814
Merge pull request #14254 from sparrowjack63/master 
Dont check Bucket Name in Nextcloud
 2019-03-01 13:46:04 +01:00
S. Cat
e3142a91b2 Bucket name - Alert in debug mode 
Alert in debug mode only when trying to mount non compatible DNS bucket name (in order to not flood the logs)

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
S. Cat
381ae44d38 Correct code indent 
Just correct code indent.

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
S. Cat
77473b0228 Warning on S3 not compatible DNS bucket name 
The new created bucket should respect the DNS compatibility, nevertheless, Nextcloud should accept to mount "old created" buckets that does not respect DNS compatibility (Backward compatibility, or compatibility with CEPH).

Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
Sebastien Cat
59c3d8d85f Dont check Bucket Name in Nextcloud 
Signed-off-by: Sebastien Cat <sparrow.jack63@gmail.com>
Signed-off-by: Sébastien Cat <sebastien.cat@inra.fr>
 2019-03-01 11:28:54 +01:00
Roeland Jago Douma
f6f002e2ad
Merge pull request #14210 from nextcloud/fix/14192/fix_empty_uploads 
Fix empty file uploads to S3 (and other streaming storages)
 2019-02-18 15:39:17 +01:00
Roeland Jago Douma
104616c1da
Use CountWrapper instead of almost identical CountReadStream 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-02-15 13:24:58 +01:00
Roeland Jago Douma
61d66d7954
Fix empty file uploads to S3 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2019-02-15 08:41:39 +01:00
Volker
d06c25b9c8
Update SwiftFactory.php 
Auth-Service (in explizit v3) will recheck the cached-token and will end in an "token expired exception".
 2019-02-13 12:26:27 +01:00
William Pain
c26bdd4731 Fix: Check if $this->params['user'] is an array 
Signed-off-by: William Pain <pain.william@gmail.com>
 2019-02-05 09:56:42 +01:00
Robin Appelman
1d322d3b37
always use multipart uploader for s3 uploads 
the multipart uploader handles non seekable streams while `upload` does not

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-01-28 14:51:19 +01:00
Morris Jobke
5a27e54f4b
Merge pull request #13032 from nextcloud/objectstore-write-exists 
upload new files in objectstore to a .part path first
 2019-01-14 11:26:29 +01:00
Robin Appelman
615fb8cd77
Cache tokens when using swift's v2 authentication 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2019-01-13 19:59:48 +01:00
Robin Appelman
d6bf5d4384
upload new files in objectstore to a .part path first 
This prevent the object store and cache from getting out of sync
when an objectstore silently fails or the php process get's killed
during the upload without giving us the chance to cleanup

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-12-12 15:24:40 +01:00
Robin Appelman
6c9f2644cf
Add objectExists to objectstore interface 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-12-12 14:17:19 +01:00
Morris Jobke
cef8eadf2a
Add PHPDoc for thrown exception 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-11-19 15:34:07 +01:00
Robin Appelman
35251928d5
forward object not found error in switch as dav 404 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-11-19 11:34:38 +01:00
Robin Appelman
93de63777e
extend storage api to allow directly writing a stream to storage 
this removes the need for temporary storages with some external storage backends.
The new method is added to a separate interface to maintain compatibility with
storage backends implementing the storage interface directly (without inheriting common)

Currently the interface is implemented for objectstorage based storages and local storage
and used by webdav uploads

Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-10-31 21:10:44 +01:00
Robin Appelman
47fd3499a3
Fix listing s3 objects when using google cloud aws compatibility 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-06-11 17:10:57 +02:00
Robin Appelman
ac26175a17
add azure unit tests with azurite 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-06-04 16:18:03 +02:00
Robin Appelman
60b7cbc0f0
add azure object storage backend 
Signed-off-by: Robin Appelman <robin@icewind.nl>
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-06-04 10:57:52 +02:00
Roeland Jago Douma
a0e038f912
generateTokenFromCache is only valid for V3 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-26 20:51:05 +02:00
Roeland Jago Douma
7318882357
json_decode an associative array for the swift cached token 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-26 20:47:47 +02:00
Arthur Schiwon
38a90130ce
move log constants to ILogger 
Signed-off-by: Arthur Schiwon <blizzz@arthur-schiwon.de>
 2018-04-26 10:45:52 +02:00
Roeland Jago Douma
1ceb081c9b
Merge pull request #9280 from nextcloud/bugfix/noid/swift-v3-scope 
Swift v3 requires setting a scope
 2018-04-24 09:49:27 +02:00
Roeland Jago Douma
ba71918ad2
Swift v3 requires setting a scope 
Unscoped auth is not guaranteed to return a catalog. So require the
scope to be set properly.

See https://developer.openstack.org/api-ref/identity/v3/index.html#password-authentication-with-unscoped-authorization

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-04-23 19:34:09 +02:00
Morris Jobke
431ccb6386
Fix undefined variable - found by phan 
Added in https://github.com/nextcloud/server/pull/8314/files#diff-a55c10804586cd797b7f3f16556c1457R157

Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-04-23 11:34:41 +02:00
Morris Jobke
2f068bc9d1
Add more logging for the object storage during creation of the buckets 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-04-05 17:26:09 +02:00
Morris Jobke
63bc633d89
Merge pull request #8857 from nextcloud/swift-verify-cached-token 
verify cached swift token
 2018-03-19 10:10:04 +01:00
Robin Appelman
41954d2903
verify cached swift token 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-03-16 15:20:16 +01:00
Roeland Jago Douma
d3e7996e80
Fix bucket and container usage 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-03-14 13:57:14 +01:00
Roeland Jago Douma
4e8d5504d2
Properly cache swift v3 token 
The V3 token has an export function. Without this the token is
effectively never cached.

Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-03-12 20:53:50 +01:00
Roeland Jago Douma
d05469959e
Merge pull request #8715 from nextcloud/objectstore-no-part-files 
disable part files for object stores
 2018-03-08 20:52:16 +01:00
Robin Appelman
fc35352122
disable part files for object stores 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-03-07 13:33:35 +01:00
Robin Appelman
9764e70494
don't read existing file when overwriting using object store 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-03-07 12:05:57 +01:00
Roeland Jago Douma
812ea354d0
User proper username + pass 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-02-26 14:54:18 +01:00
Robin Appelman
31dd6d6aae
add support for v3 swift auth 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-02-26 14:54:08 +01:00
Robin Appelman
817f2f8f49
adjust swift external storage to new openstack sdk 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-02-19 19:41:16 +01:00
Robin Appelman
3192efc415
adjust swift object storage to new openstack sdk 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-02-19 19:41:16 +01:00
Morris Jobke
d3d045dd5c
Remove unused import statements 
Signed-off-by: Morris Jobke <hey@morrisjobke.de>
 2018-02-14 16:55:43 +01:00
Roeland Jago Douma
3e0ef7e3f2
Try to silence phan 
Signed-off-by: Roeland Jago Douma <roeland@famdouma.nl>
 2018-02-08 15:18:26 +01:00
Robin Appelman
e34c20af7d
don't swallow exception from touch 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-02-08 15:18:26 +01:00
Robin Appelman
85d23dd6fc
better error message on catalog error 
Signed-off-by: Robin Appelman <robin@icewind.nl>
 2018-02-08 15:18:26 +01:00