Since the library can only store the full response in memory on
download, we use an alternate client lib and set the correct headers to
be able to stream the content to a temp file.
Using the Guzzle stream directly here will only return 1739 characters for `fread` instead of all data. This leads to the problem that the stream is read incorrectly and thus the data cannot be properly decrypted => 💣
This approach copies the data into a local temporary file, as done before in all stable releases as well as other storage connectors.
While this approach will load the whole file into memory, this is already was has happened before in any stable release as well. See d608c37c90 for the breaking change.
To test this enable Google Drive as external storage and upload some files with encryption enabled. Reading the file should fail now.
Fixes https://github.com/owncloud/core/issues/22590
The current logic is checking whether:
1. The returned value is a boolen
2. The returned value is a string and then matches for "true"
Since the config is now written to the database the data is now a string with the value "1" if HTTPS is set to true. Effectively this option was thus always disabled at the moment, falling back to plain HTTP.
This change casts the data to a boolean if it is defined as boolean.
Fixes https://github.com/owncloud/core/issues/22605
Fixes https://github.com/owncloud/core/issues/22016
When encryption is enabled, GDrive would think that all files are text
files. This fix falls back to the extension based detection when a
non-special mime type is returned
1) Properly detect empty file extension, can be null.
2) When renaming part file to final file, use the correct file name
without extension, if it exists
3) When renaming a file, do not delete the original file if it had the
same id, which can happen with part files
The API library does not support streaming and always reads the full
file into memory.
This workaround copies the signed headers to a Guzzle request and
returns the response as stream.
Makes sure that the paths are trimmed to avoid duplicate entries like
"/test" and "test". This should make this storage slightly faster by
reducing the cache misses.
StoragesService::getStorages() will check the visibility of the backend
and auth mechanism for the storage, and if either are not visible to the
user (aka disabled by admin) then the storage will be filtered out. The
original method StoragesService::getAllStorages() still exists in case
such storages need to be detected, but its use is discouraged.
VisibilityTrait -> PermissionsTrait
PermissionsTrait stores two sets of data, $permissions and
$allowedPermissions (analogous to $visibility and $allowedVisibility of
VisibilityTrait). Each set is a map of user type ('admin' or 'personal')
to permissions (mounting permission, create permission).
The result is that a backend can now be restricted for creation, while
still allowing it to be mounted. This is useful for deprecating backends
or auth mechanisms, preventing new storages being created, while still
allowing existing storages to be mounted.
Introduces the basic password authentication mechanism, along with a
mechanism based on ownCloud credentials stored in the user session.
Change to lib/private is an extension of PermissionsMask, as
isSharable() override was missing.
Session credentials auth mechanism now disables sharing on applied
storages, as credentials will not be available.
Failure to prepare the storage during backend or auth mechanism
manipulation will throw an InsufficientDataForMeaningfulAnswerException,
which is propagated to StorageNotAvailableException in the filesystem
layer via the FailedStorage helper class.
When a storage is unavailable not due to failure, but due to
insufficient data being available, a special 'indeterminate' status is
returned to the configuration UI.
The following functions have been removed:
- addMountPoint()
- removeMountPoint()
- movePersonalMountPoint()
registerBackend() has been rewritten as a shim around BackendService,
allowing legacy code to interact with the new API seamlessly
addMountPoint() was already disconnected from all production code, so
this commit completes the job and removes the function itself, along
with disconnecting and removing related functions. Unit tests have
likewise been removed.
getAbsoluteMountPoints(), getSystemMountPoints() and
getPersonalMountPoints() have been rewritten to use the StoragesServices
Prior to this, the storage class name was stored in mount.json under the
"class" parameter, and the auth mechanism class name under the
"authMechanism" parameter. This decouples the class name from the
identifier used to retrieve the backend or auth mechanism.
Now, backends/auth mechanisms have a unique identifier, which is saved in
the "backend" or "authMechanism" parameter in mount.json respectively.
An identifier is considered unique for the object it references, but the
underlying class may change (e.g. files_external gets pulled into core
and namespaces are modified).
A backend can now specify generic authentication schemes that it
supports, instead of specifying the parameters for its authentication
method directly. This allows multiple authentication mechanisms to be
implemented for a single scheme, providing altered functionality.
This commit introduces the backend framework for this feature, and so at
this point the UI will be broken as the frontend does not specify the
required information.
Terminology:
- authentication scheme
Parameter interface for the authentication method. A backend
supporting the 'password' scheme accepts two parameters, 'user' and
'password'.
- authentication mechanism
Specific mechanism implementing a scheme. Basic mechanisms may
forward configuration options directly to the backend, more advanced
ones may lookup parameters or retrieve them from the session
New dropdown selector for external storage configurations to select the
authentication mechanism to be used.
Authentication mechanisms can have visibilities, just like backends.
The API was extended too to make it easier to add/remove visibilities.
In addition, the concept of 'allowed visibility' has been introduced, so
a backend/auth mechanism can force a maximum visibility level (e.g.
Local storage type) that cannot be overridden by configuration in the
web UI.
An authentication mechanism is a fully instantiated implementation. This
allows an implementation to have dependencies injected into it, e.g. an
\OCP\IDB for database operations.
When a StorageConfig is being prepared for mounting, the authentication
mechanism implementation has manipulateStorage() called,
which inserts the relevant authentication method options into the
storage ready for mounting.
UserGlobalStoragesService reads the global storage configuration,
cherry-picking storages applicable to a user. Writing storages through
this service is forbidden, on punishment of throwing an exception.
Storage IDs may also be config hashes when retrieved from this service,
as it is unable to update the storages with real IDs.
As UserGlobalStoragesService and UserStoragesService share a bit of code
relating to users, that has been split into UserTrait. UserTrait also
allows for the user set to be overridden, rather than using the user
from IUserSession.
Config\ConfigAdapter has been reworked to use UserStoragesService and
UserGlobalStoragesService instead of
OC_Mount_Config::getAbsoluteMountPoints(), further reducing dependance
on that horrible static class.
Backends are registered to the BackendService through new data
structures:
Backends are concrete classes, deriving from
\OCA\Files_External\Lib\Backend\Backend. During construction, the
various configuration parameters of the Backend can be set, in a design
similar to Symfony Console.
DefinitionParameter stores a parameter configuration for an external
storage: name of parameter, human-readable name, type of parameter
(text, password, hidden, checkbox), flags (optional or not).
Storages in the StoragesController now get their parameters validated
server-side (fixes a TODO).
Storage status is saved in the database. Failed storages are rechecked every
10 minutes, while working storages are rechecked every request.
Using the files_external app will recheck all external storages when the
settings page is viewed, or whenever an external storage is saved.
Explicitly clear the stat cache after deleting an empty folder to make
sure it is properly detected as deleted in subsequent requests.
This works around a problem with phpseclib where the folder is properly
deleted remotely but the stat cache was not updated.
The code was missing the "MetadataDirective".
Once added, some other parts of the code failed because the format of mtime was wrong.
So this PR uses the RFC format that the S3 library already uses.
Additionally, the code path where mtime is null was missing. Now defaulting to
the current time.
str_replace for $user substitution was converting the data type of
mountOptions to string. This fix prevents this to happen by making sure
only strings are processed by substitution.
Also added a int conversion when reading the watcher policy
When reading in old mount.json files, they do not contain config ids.
Since these are needed to be able to use the UI and the new service
classes, these will be generated automatically.
The config grouping is based on a config hash.
The mount options are now passed to the UI and stored in a hidden field.
The ext storage controllers and services also know how to load/save them
from the legacy config.
- Added StorageConfig class to replace ugly arrays
- Implemented StorageService and StorageController for Global and User
storages
- Async status checking for storages (from Xenopathic)
- Auto-generate id for external storage configs (not the same as
storage_id)
- Refactor JS classes for external storage settings, this mostly
moves/encapsulated existing global event handlers into the
MountConfigListView class.
- Added some JS unit tests for the external storage UI
