Commit graph

612 commits

Author SHA1 Message Date
Robin McCorkell
cc88c5f4b8 Implement more fine-grained external storage permissions model
VisibilityTrait -> PermissionsTrait

PermissionsTrait stores two sets of data, $permissions and
$allowedPermissions (analogous to $visibility and $allowedVisibility of
VisibilityTrait). Each set is a map of user type ('admin' or 'personal')
to permissions (mounting permission, create permission).

The result is that a backend can now be restricted for creation, while
still allowing it to be mounted. This is useful for deprecating backends
or auth mechanisms, preventing new storages being created, while still
allowing existing storages to be mounted.
2015-08-28 17:21:58 +01:00
Robin Appelman
d14252d9c7 make sure we actually have an object store 2015-08-28 16:13:19 +02:00
Robin McCorkell
080fafe63a AjaxController uses RSA auth mechanism 2015-08-28 12:58:47 +01:00
Robin McCorkell
1084e3adc7 Migrate SFTP_Key external storage to new API
The SFTP backend now supports public key authentication alongside
password authentication.
2015-08-28 12:58:47 +01:00
Robin McCorkell
cb1ef82702 Migrate SMB_OC external storage to new API
SMB_OC has been merged with SMB, via the identifier aliases mechanism.
Legacy migration is done to the Session Credentials password mechanism
2015-08-28 12:58:47 +01:00
Robin McCorkell
19bc5a452a Migrate Swift external storage to new API
The Rackspace/OpenStack differences have been split into separate auth
mechanisms, with correct legacy migration
2015-08-28 12:58:47 +01:00
Morris Jobke
494c1d7417 Merge pull request #18440 from owncloud/ext-backends.customjs
Migrate custom JS external storage backends to new registration API [part 2]
2015-08-25 11:52:56 +02:00
Morris Jobke
424759908d Merge pull request #18445 from owncloud/ext-only-setuservars-string
setUserVars() should only attempt substitution with strings
2015-08-25 08:43:01 +02:00
Robin McCorkell
88a78237b0 Migrate Google external storage to new API 2015-08-25 00:22:10 +01:00
Robin McCorkell
a50ef61876 Migrate Dropbox external storage to new API 2015-08-25 00:22:10 +01:00
Robin McCorkell
ced04f9ad2 Migrate AmazonS3 external storage to new API 2015-08-25 00:22:10 +01:00
Robin McCorkell
6379b1932f Allow domain to be specified for SMB 2015-08-21 10:30:42 +01:00
Robin McCorkell
5ded5c6fc1 Add availability methods to files_external FailedStorage 2015-08-20 22:47:57 +01:00
Robin McCorkell
d93bf35482 Merge pull request #18408 from owncloud/ext-ocp
Use OCP classes as much as possible in files_external v2
2015-08-20 13:09:12 +01:00
Robin McCorkell
643e3a5b6d Convert string booleans to real booleans
Legacy compatibility, from the days in stable8 when checkbox boolean
values were stored as the strings 'true' and 'false'.
2015-08-20 12:23:12 +01:00
Robin McCorkell
62d328525a setUserVars() should only attempt substitution with strings 2015-08-20 02:19:03 +01:00
Robin McCorkell
8cd47e4f03 Fix 'Declaration of SessionStorageWrapper::__construct()...' 2015-08-19 20:14:56 +01:00
Morris Jobke
b3356b1288 Merge pull request #18432 from owncloud/ext-backends.simple
Migrate simple external storage backends to new registration API [part 1]
2015-08-19 20:04:20 +02:00
Robin McCorkell
d577aad4ac Use OCP classes as much as possible in files_external 2015-08-19 15:26:38 +01:00
Robin Appelman
273c776eb7 add missing return statements in getSystem/getPersonalMountPoints 2015-08-19 16:26:33 +02:00
Robin McCorkell
16389270ff Migrate SFTP external storage to new API 2015-08-19 14:41:44 +01:00
Robin McCorkell
a99e524898 Migrate OwnCloud external storage to new API 2015-08-19 14:41:43 +01:00
Robin McCorkell
68418bdd34 Migrate DAV external storage to new API 2015-08-19 14:41:43 +01:00
Robin McCorkell
ab8c738b8d Migrate SMB external storage to new API 2015-08-19 14:41:43 +01:00
Robin McCorkell
0f1809eced Migrate FTP external storage to new API 2015-08-19 14:41:43 +01:00
Robin McCorkell
0ffb51c6cc Migrate Local external storage to new API 2015-08-19 14:41:43 +01:00
Robin McCorkell
3bb793b6a7 Implement password authentication mechanisms
Introduces the basic password authentication mechanism, along with a
mechanism based on ownCloud credentials stored in the user session.

Change to lib/private is an extension of PermissionsMask, as
isSharable() override was missing.

Session credentials auth mechanism now disables sharing on applied
storages, as credentials will not be available.
2015-08-19 14:20:09 +01:00
Robin McCorkell
b6eb952ac6 Propagate auth mechanism/backend failures to filesystem layer
Failure to prepare the storage during backend or auth mechanism
manipulation will throw an InsufficientDataForMeaningfulAnswerException,
which is propagated to StorageNotAvailableException in the filesystem
layer via the FailedStorage helper class.

When a storage is unavailable not due to failure, but due to
insufficient data being available, a special 'indeterminate' status is
returned to the configuration UI.
2015-08-19 10:08:23 +01:00
Robin McCorkell
c592e24c87 Make Application a singleton
The same Application must be used in the settings templates and in
routes, so that any registered backends are correctly seen
2015-08-19 10:08:23 +01:00
Robin McCorkell
97dbc79c16 Compatibility shims for OC_Mount_Config
The following functions have been removed:
 - addMountPoint()
 - removeMountPoint()
 - movePersonalMountPoint()

registerBackend() has been rewritten as a shim around BackendService,
allowing legacy code to interact with the new API seamlessly

addMountPoint() was already disconnected from all production code, so
this commit completes the job and removes the function itself, along
with disconnecting and removing related functions. Unit tests have
likewise been removed.

getAbsoluteMountPoints(), getSystemMountPoints() and
getPersonalMountPoints() have been rewritten to use the StoragesServices
2015-08-19 10:08:14 +01:00
Robin McCorkell
1eeca031f8 Split backend identifiers from the class name
Prior to this, the storage class name was stored in mount.json under the
"class" parameter, and the auth mechanism class name under the
"authMechanism" parameter. This decouples the class name from the
identifier used to retrieve the backend or auth mechanism.

Now, backends/auth mechanisms have a unique identifier, which is saved in
the "backend" or "authMechanism" parameter in mount.json respectively.
An identifier is considered unique for the object it references, but the
underlying class may change (e.g. files_external gets pulled into core
and namespaces are modified).
2015-08-19 10:05:11 +01:00
Robin McCorkell
272a46ebe1 Authentication mechanisms for external storage backends
A backend can now specify generic authentication schemes that it
supports, instead of specifying the parameters for its authentication
method directly. This allows multiple authentication mechanisms to be
implemented for a single scheme, providing altered functionality.

This commit introduces the backend framework for this feature, and so at
this point the UI will be broken as the frontend does not specify the
required information.

Terminology:
 - authentication scheme
    Parameter interface for the authentication method. A backend
    supporting the 'password' scheme accepts two parameters, 'user' and
    'password'.
 - authentication mechanism
    Specific mechanism implementing a scheme. Basic mechanisms may
    forward configuration options directly to the backend, more advanced
    ones may lookup parameters or retrieve them from the session

New dropdown selector for external storage configurations to select the
authentication mechanism to be used.

Authentication mechanisms can have visibilities, just like backends.
The API was extended too to make it easier to add/remove visibilities.
In addition, the concept of 'allowed visibility' has been introduced, so
a backend/auth mechanism can force a maximum visibility level (e.g.
Local storage type) that cannot be overridden by configuration in the
web UI.

An authentication mechanism is a fully instantiated implementation. This
allows an implementation to have dependencies injected into it, e.g. an
\OCP\IDB for database operations.

When a StorageConfig is being prepared for mounting, the authentication
mechanism implementation has manipulateStorage() called,
which inserts the relevant authentication method options into the
storage ready for mounting.
2015-08-19 10:05:11 +01:00
Robin McCorkell
a6a69ef1df Introduce UserGlobalStoragesService
UserGlobalStoragesService reads the global storage configuration,
cherry-picking storages applicable to a user. Writing storages through
this service is forbidden, on punishment of throwing an exception.
Storage IDs may also be config hashes when retrieved from this service,
as it is unable to update the storages with real IDs.

As UserGlobalStoragesService and UserStoragesService share a bit of code
relating to users, that has been split into UserTrait. UserTrait also
allows for the user set to be overridden, rather than using the user
from IUserSession.

Config\ConfigAdapter has been reworked to use UserStoragesService and
UserGlobalStoragesService instead of
OC_Mount_Config::getAbsoluteMountPoints(), further reducing dependance
on that horrible static class.
2015-08-19 10:05:11 +01:00
Robin McCorkell
37beb58c6f Introduce BackendService for managing external storage backends
Backends are registered to the BackendService through new data
structures:

Backends are concrete classes, deriving from
\OCA\Files_External\Lib\Backend\Backend. During construction, the
various configuration parameters of the Backend can be set, in a design
similar to Symfony Console.

DefinitionParameter stores a parameter configuration for an external
storage: name of parameter, human-readable name, type of parameter
(text, password, hidden, checkbox), flags (optional or not).

Storages in the StoragesController now get their parameters validated
server-side (fixes a TODO).
2015-08-19 10:05:11 +01:00
Robin McCorkell
214729a552 Merge pull request #14779 from owncloud/use-iterator-directory
Use the new IteratorDirectory instead of the fakedir wrapper
2015-08-07 22:16:47 +01:00
Vincent Petry
b3a1aef934 Merge pull request #13641 from owncloud/cache-storage-status
Store storage availability in database
2015-08-07 17:31:03 +02:00
Vincent Petry
4c9f55a325 Merge pull request #17680 from owncloud/ftp-rmdir-file
handle rmdir on files for ftp storages
2015-08-06 18:05:28 +02:00
Lukas Reschke
99224905d8 Use absolute namespace 2015-08-05 10:30:39 +02:00
Andreas Fischer
3b803db6d8 Update phpseclib to 2.0 2015-08-03 09:43:45 +02:00
Vincent Petry
761ba344b7 Double slash for SMB storage id for compatibility 2015-07-28 11:41:54 +02:00
Robin McCorkell
df19cabb44 Store storage availability in database
Storage status is saved in the database. Failed storages are rechecked every
10 minutes, while working storages are rechecked every request.

Using the files_external app will recheck all external storages when the
settings page is viewed, or whenever an external storage is saved.
2015-07-20 16:27:26 +01:00
Robin Appelman
db89d1cec8 handle rmdir on files for ftp storages 2015-07-16 15:44:10 +02:00
Robin Appelman
da951ba059 Use the new IteratorDirectory instead of the fakedir wrapper 2015-07-15 13:57:53 +02:00
Thomas Müller
079e9cecba Revert "Use OCP classes as much as possible in files_external" 2015-07-02 13:00:21 +02:00
Robin McCorkell
728a22cda1 Use OCP classes as much as possible in files_external 2015-07-01 09:08:28 +02:00
Morris Jobke
044d2ece07 Merge pull request #15506 from rullzer/core_apps_oc_log2ocp_util
Move core apps from OC_Log::write to OCP\Util
2015-07-01 08:53:16 +02:00
Morris Jobke
f63915d0c8 update license headers and authors 2015-06-25 14:13:49 +02:00
Thomas Müller
25581c7b63 Merge pull request #16940 from owncloud/ext-s3-touchmtimefix
Properly set mtime on S3 for touch operation
2015-06-22 22:25:45 +02:00
Vincent Petry
e15dd783ab Workaround for empty dir deletion for SFTP
Explicitly clear the stat cache after deleting an empty folder to make
sure it is properly detected as deleted in subsequent requests.

This works around a problem with phpseclib where the folder is properly
deleted remotely but the stat cache was not updated.
2015-06-18 17:40:38 +02:00
Vincent Petry
f98030020f Properly set mtime on S3 for touch operation
The code was missing the "MetadataDirective".
Once added, some other parts of the code failed because the format of mtime was wrong.
So this PR uses the RFC format that the S3 library already uses.

Additionally, the code path where mtime is null was missing. Now defaulting to
the current time.
2015-06-15 17:35:09 +02:00
Vincent Petry
d3f828af45 Skip directory entry in S3 opendir
The result set contains the directory itself, so skip it to avoid
scanning a non-existing directory
2015-06-15 17:30:09 +02:00
Roeland Jago Douma
7a8072e958 Move core apps from OC_Log::write to OCP\Util 2015-05-18 10:57:52 +02:00
Thomas Müller
a637bd7f2b Avoid function name collision in dropbox external storage 2015-04-20 14:56:51 +02:00
Vincent Petry
a6c5d65fd6 Fix SFTP storage id to be compatible with older ids
Remove port from SFTP storage id if it is 22.
This will prevent recreating a different storage entry due to id
mismatch after upgrade.
2015-04-15 16:25:47 +02:00
Lukas Reschke
c3c50fb481 Fix language level incompatibilties
Arbitrary expressions in empty are allowed in PHP 5.5 ands upwards. Seems to only affect master.

Fixed https://github.com/owncloud/core/issues/15463
2015-04-08 13:55:10 +02:00
Lukas Reschke
c052ee7584 Only escape single quotes 2015-03-31 15:30:49 +02:00
Vincent Petry
7036309e22 Added rawurlencode for other params in GDrive storage 2015-03-31 15:14:08 +02:00
Vincent Petry
5c9998179f Properly quote file names in listFiles query for GDrive 2015-03-31 15:14:03 +02:00
Lukas Reschke
53f67fc65d Revert "Properly quote file names in listFiles query for GDrive" 2015-03-31 15:12:35 +02:00
Frank Karlitschek
b216b3fad0 Merge pull request #15215 from owncloud/ext-gdrivelistfilequoting
Properly quote file names in listFiles query for GDrive
2015-03-31 09:08:59 -04:00
Robin McCorkell
308a43a10e Merge pull request #15076 from owncloud/fix-smb-oc-admin
Support constructing an SMB_OC without saved session credentials
2015-03-30 15:38:22 +01:00
Robin McCorkell
c2909355bf Merge pull request #14925 from owncloud/ext-mountoptions-ui
Mount options GUI for external storage
2015-03-26 21:36:34 +00:00
Vincent Petry
58b4c2c0e5 Make sure mountOptions keep their data type
str_replace for $user substitution was converting the data type of
mountOptions to string. This fix prevents this to happen by making sure
only strings are processed by substitution.

Also added a int conversion when reading the watcher policy
2015-03-26 19:24:37 +01:00
Vincent Petry
534892d456 Added rawurlencode for other params in GDrive storage 2015-03-26 13:57:28 +01:00
Vincent Petry
43588fe2f7 Properly quote file names in listFiles query for GDrive 2015-03-26 12:15:02 +01:00
Jenkins for ownCloud
b585d87d9d Update license headers 2015-03-26 11:44:36 +01:00
Robin McCorkell
1f3fdf17ef Support constructing an SMB_OC without saved session credentials 2015-03-20 16:24:30 +00:00
Vincent Petry
e44ab2dcfe Merge pull request #11884 from owncloud/ext-appframework
Cleanup ext storage CRUD code, introduce storage config id
2015-03-20 14:17:32 +01:00
Robin McCorkell
c37913b1d5 Introduce async status checking 2015-03-20 09:48:14 +00:00
Vincent Petry
e5e30924b1 Fix PHPDoc in files_external, add missing tag
Fix various PHPDoc issues in external storage app.

Added missing NoAdminRequired tag
2015-03-17 11:42:52 +01:00
Vincent Petry
5b6ecb2d98 Fix root request path for Dropbox 2015-03-17 10:56:24 +01:00
Vincent Petry
72632ad402 Generate storage config ids when missing
When reading in old mount.json files, they do not contain config ids.
Since these are needed to be able to use the UI and the new service
classes, these will be generated automatically.

The config grouping is based on a config hash.
2015-03-16 12:18:01 +01:00
Vincent Petry
fb4cf53253 Fix mount config unit tests
Fix expected result now that it returns the status.
Added isset for some properties that are not always present.
2015-03-13 17:43:38 +01:00
Vincent Petry
34c8b1ac77 Mount options for ext storage are now passed around
The mount options are now passed to the UI and stored in a hidden field.
The ext storage controllers and services also know how to load/save them
from the legacy config.
2015-03-13 12:49:11 +01:00
Thomas Müller
5855108e9b drop any fallback code related to curl - refs https://github.com/owncloud/core/pull/14838#issuecomment-78586447 2015-03-12 21:43:41 +01:00
Vincent Petry
ce94a998dd Use storage id + appframework for ext storage CRUD
- Added StorageConfig class to replace ugly arrays
- Implemented StorageService and StorageController for Global and User
  storages
- Async status checking for storages (from Xenopathic)
- Auto-generate id for external storage configs (not the same as
  storage_id)
- Refactor JS classes for external storage settings, this mostly
  moves/encapsulated existing global event handlers into the
  MountConfigListView class.
- Added some JS unit tests for the external storage UI
2015-03-12 18:51:02 +01:00
Morris Jobke
06aef4e8b1 Revert "Updating license headers"
This reverts commit 6a1a4880f0.
2015-02-26 11:37:37 +01:00
Thomas Müller
f5b6226732 Merge pull request #14462 from owncloud/add-some-php-docs
Add some PHPDocs
2015-02-25 02:53:22 -08:00
Thomas Müller
f6044fe429 Merge pull request #14460 from owncloud/revive/12971
Add support for sftp custom port
2015-02-25 02:52:25 -08:00
Robin McCorkell
edfeb05a05 Fix bugs in ownCloud external storage 2015-02-25 10:10:13 +00:00
Lukas Reschke
dbb665a361 Add some PHPDocs
My IDE was crying...
2015-02-24 13:44:40 +01:00
Lukas Reschke
1229184873 Fix code that never worked
Introduced with 1df8a5a774 in February 2013…
2015-02-24 13:26:23 +01:00
Lukas Reschke
6dd5bc0379 Fix code style as per review 2015-02-24 13:22:12 +01:00
SA
350f8eb86e Add support for sftp custom port 2015-02-24 13:15:38 +01:00
Jenkins for ownCloud
6a1a4880f0 Updating license headers 2015-02-23 12:13:59 +01:00
Andreas Fischer
c640e2fa66 Update phpseclib to version 0.3.9. 2015-02-18 01:16:23 +01:00
Robin Appelman
6467369cad make sure the root has a trailing slash 2015-02-16 13:52:11 +01:00
Robin Appelman
e98983485f handle deleting non existing files 2015-02-16 13:52:11 +01:00
Robin Appelman
e5731b6e1a Use better way to check for the existence of smbclient 2015-02-16 13:52:11 +01:00
Robin Appelman
d2255a1d30 New SMB storage backend 2015-02-16 13:52:11 +01:00
Vincent Petry
bd01ff135a Merge pull request #13190 from is-apps/master-sftp-key
Add SFTP public key authentication support
2015-02-10 16:44:29 +01:00
Vincent Petry
3764515365 Allow saving incomplete external storage config
This is needed for Dropbox and others that need a token.
2015-01-27 13:44:05 +01:00
Ross Nicoll
64f4f8fc84 Add support for SFTP key authentication
Add support for external files accessed via SFTP using public key exchange authentication.
Keys are generated automatically when the configuration is added, or can be regenerated on demand if a key is compromised.
Creation of a new configuration row now triggers focus on that row. This is used to trigger auto-configuration for SFTP keys.
Generated public keys are saved in user's data directory for easy retrieval by an external application.
Add controller for SFTP key generation AJAX
SFTP class initialisation no longer produces a warning if the password field is missing.
Add unit tests for SFTP with key authentication backend
2015-01-14 17:00:34 +00:00
Morris Jobke
75e8b39826 Merge pull request #12879 from owncloud/mountconfig
Add mount specific options
2014-12-18 23:38:14 +01:00
Jörn Friedrich Dreyer
49318b4d93 Merge pull request #12896 from owncloud/drop-deprecated
first step to drop \OCP\Config:: in favour of IConfig
2014-12-18 23:01:49 +01:00
Robin Appelman
d075b1d743 Add the storage id to the mount config 2014-12-17 14:22:03 +01:00
Robin Appelman
95a145f67f Load mount specific options from the mount config 2014-12-17 14:03:50 +01:00
Morris Jobke
d41082f4d6 first step to drop \OCP\Config:: in favour of IConfig 2014-12-17 11:12:37 +01:00
Lukas Reschke
5eaa2214b8 Remove conditional check for 5.3 compatibility 2014-12-12 12:25:59 +01:00
Morris Jobke
e3de51078d Merge pull request #12090 from AdamWill/google-rename-delete
google: delete original after successful rename
2014-12-10 09:10:17 +01:00