Merge branch 'release/4.6.0'

This commit is contained in:
James Cole 2017-06-28 18:13:00 +02:00
commit 63be574a14
142 changed files with 5229 additions and 5812 deletions

View File

@ -4,20 +4,64 @@
## Feature requests
If you are requesting a new feature, please check out the list of [often requested features](https://firefly-iii.github.io/requested-features/).
I am always interested in expanding Firefly III's many features. If you are requesting a new feature, please check out the list of [often requested features](https://firefly-iii.github.io/requested-features/).
## Bugs
If you find a bug, please take the time and see if the [demo site](https://firefly-iii.nder.be/) is also suffering from this bug. Include as many log files and details as you think are necessary.
First of all: thank you for reporting a bug instead of ditching the tool altogether. If you find a bug, please take the time and see if the [demo site](https://firefly-iii.nder.be/) is also suffering from this bug. Include as many log files and details as you think are necessary. Bugs have a lot of priority!
## Installation problems
Take the time to read the [installation guide FAQ](https://firefly-iii.github.io/installation-guide-faq/) and make sure you search through closed issues for the problems other people have had. Your problem may be among them!
Please take the time to read the [installation guide FAQ](https://firefly-iii.github.io/installation-guide-faq/) and make sure you search through closed issues for the problems other people have had. Your problem may be among them! If not, open an issue and I will help where I can.
## Pull requests
I can only accept pull requests against the `develop` branch, never the `master` branch.
When contributing to Firefly III, please first discuss the change you wish to make via issue, email, or any other method. I can only accept pull requests against the `develop` branch, never the `master` branch.
## Translations :us: :fr: :de:
If you see a spelling error, grammatical error or a weird translation in your language, please join [our CrowdIn](https://crowdin.com/project/firefly-iii) project. There, you can submit your translations and fixes. The GitHub repository will download these automatically and they will be included in the next release.
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as contributors and maintainers pledge to making participation in our project and our community a harassment-free experience for everyone, regardless of age, body size, disability, ethnicity, gender identity and expression, level of experience, nationality, personal appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable behavior and are expected to take appropriate and fair corrective action in response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, or to ban temporarily or permanently any contributor for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces when an individual is representing the project or its community. Examples of representing a project or community include using an official project e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event. Representation of a project may be further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team at thegrumpydictator@gmail.com. All complaints will be reviewed and investigated and will result in a response that is deemed necessary and appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant](http://contributor-covenant.org/), version 1.4, available at [http://contributor-covenant.org/version/1/4](http://contributor-covenant.org/version/1/4).

View File

@ -16,7 +16,7 @@ install:
- php artisan optimize
- php artisan env
- cp .env.testing .env
- mv storage/database/databasecopy.sqlite storage/database/database.sqlite
- wget -q https://github.com/firefly-iii/test-data/raw/master/storage/database.sqlite -O storage/database/database.sqlite
- mkdir -p build/logs
script:

View File

@ -2,7 +2,24 @@
All notable changes to this project will be documented in this file.
This project adheres to [Semantic Versioning](http://semver.org/).
## [4.5.0] - 2017-07-07
## [4.6.0] - 2017-06-28
### Changed
- Revamped import routine. Will be buggy.
### Fixed
- Issue #667, postgresql reported by @skibbipl.
- Issue #680 by @Xeli
- Fixed #660
- Fixes #672, reported by @dzaikos
- Translation error fixed by
- Fix a bug where the balance routine forgot to account for accounts without a currency preference.
- Various other bugfixes.
### Security
- Initial release.
## [4.5.0] - 2017-06-07
### Added
- Better support for multi-currency transactions and display of transactions, accounts and everything. This requires a database overhaul (moving the currency information to specific transactions) so be careful when upgrading.
@ -11,7 +28,7 @@ This project adheres to [Semantic Versioning](http://semver.org/).
- Expanded Docker to work with postgresql as well, thanks to @kressh
### Fixed
- PostgreSQL support in database upgrade routine (#644, reported by @)
- PostgreSQL support in database upgrade routine (#644, reported by @skibbipl)
- Frontpage budget chart was off, fix by @nhaarman
- Was not possible to remove opening balance.

View File

@ -12,11 +12,9 @@
[![Deploy](https://www.herokucdn.com/deploy/button.svg)](https://heroku.com/deploy?template=https://github.com/firefly-iii/firefly-iii/tree/master)
Firefly III can be run on Heroku. Register for a free Heroku account and instantly run Firefly III on your very own cloud instance.
Firefly III can be run on Heroku. Register for a free Heroku account and instantly run Firefly III on your very own cloud instance. There is also a [demo site](https://firefly-iii.nder.be) with an example financial administration already present.
There is also a [demo site](https://firefly-iii.nder.be) with an example financial administration already present.
## Installation
## Getting started
To install Firefly III, you'll need a web server (preferrably on Linux) and access to the command line. Then, please read the [installation guide](https://firefly-iii.github.io/using-installing.html).
@ -26,7 +24,7 @@ Personal financial management is pretty difficult, and everybody has their own a
Firefly works on the principle that if you know where you're money is going, you can stop it from going there.
#### Some advantages of using Firefly
### Some advantages of using Firefly
- Firefly can import any CSV file, so migrating from other systems is easy.
- Firefly runs on your own server, so you are fully in control of your data. Remember, there is no such thing as "the cloud", its just somebody elses computer!
@ -35,6 +33,25 @@ Firefly works on the principle that if you know where you're money is going, you
Firefly is pretty awesome. [You can read more about Firefly III, and its features, on the Github Pages](https://firefly-iii.github.io/).
### Contributing
Please read [CONTRIBUTING.md](https://github.com/firefly-iii/firefly-iii/blob/master/.github/CONTRIBUTING.md) for details on the code of conduct, and the process for submitting pull requests.
### Versioning
We use [SemVer](http://semver.org/) for versioning. For the versions available, see [the tags](https://github.com/firefly-iii/firefly-iii/tags) on this repository.
### Authors
* James Cole
* Over time, [many people have contributed to Firefly III](https://github.com/firefly-iii/firefly-iii/graphs/contributors).
### License
This work [is licensed](https://github.com/firefly-iii/firefly-iii/blob/master/LICENSE) under a [Creative Commons Attribution-ShareAlike 4.0 International License](https://creativecommons.org/licenses/by-sa/4.0/).
### Other stuff
If you like Firefly and if it helps you save lots of money, why not send me [a dime for every dollar saved](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=44UKUT455HUFA) (this is a joke, although the Paypal form works just fine, try it!)
If you want to contact me, please open an issue or [email me](mailto:thegrumpydictator@gmail.com).

View File

@ -88,7 +88,7 @@ class CreateImport extends Command
$this->line('Stored import data...');
$job->configuration = $configurationData;
$job->status = 'settings_complete';
$job->status = 'configured';
$job->save();
$this->line('Stored configuration...');

View File

@ -13,12 +13,13 @@ declare(strict_types=1);
namespace FireflyIII\Console\Commands;
use FireflyIII\Import\ImportProcedure;
use FireflyIII\Import\Logging\CommandHandler;
use FireflyIII\Import\Routine\ImportRoutine;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\TransactionJournal;
use Illuminate\Console\Command;
use Illuminate\Support\Collection;
use Illuminate\Support\MessageBag;
use Log;
/**
@ -75,15 +76,18 @@ class Import extends Command
$monolog = Log::getMonolog();
$handler = new CommandHandler($this);
$monolog->pushHandler($handler);
$importProcedure = new ImportProcedure;
$result = $importProcedure->runImport($job);
// display result to user:
$this->presentResults($result);
$this->line('The import has completed.');
/** @var ImportRoutine $routine */
$routine = app(ImportRoutine::class);
$routine->setJob($job);
$routine->run();
// get any errors from the importer:
$this->presentErrors($job);
/** @var MessageBag $error */
foreach ($routine->errors as $index => $error) {
$this->error(sprintf('Error importing line #%d: %s', $index, $error));
}
$this->line(sprintf('The import has finished. %d transactions have been imported out of %d records.', $routine->journals->count(), $routine->lines));
return;
}
@ -96,12 +100,14 @@ class Import extends Command
private function isValid(ImportJob $job): bool
{
if (is_null($job)) {
Log::error('This job does not seem to exist.');
$this->error('This job does not seem to exist.');
return false;
}
if ($job->status != 'settings_complete') {
if ($job->status !== 'configured') {
Log::error(sprintf('This job is not ready to be imported (status is %s).', $job->status));
$this->error('This job is not ready to be imported.');
return false;
@ -109,36 +115,4 @@ class Import extends Command
return true;
}
/**
* @param ImportJob $job
*/
private function presentErrors(ImportJob $job)
{
$extendedStatus = $job->extended_status;
if (isset($extendedStatus['errors']) && count($extendedStatus['errors']) > 0) {
$this->line(sprintf('The following %d error(s) occured during the import:', count($extendedStatus['errors'])));
foreach ($extendedStatus['errors'] as $error) {
$this->error($error);
}
}
}
/**
* @param Collection $result
*/
private function presentResults(Collection $result)
{
/**
* @var int $index
* @var TransactionJournal $journal
*/
foreach ($result as $index => $journal) {
if (!is_null($journal->id)) {
$this->line(sprintf('Line #%d has been imported as transaction #%d.', $index, $journal->id));
continue;
}
$this->error(sprintf('Could not store line #%d', $index));
}
}
}

View File

@ -316,6 +316,7 @@ class UpgradeDatabase extends Command
$notification = '%s #%d uses %s but should use %s. It has been updated. Please verify this in Firefly III.';
$transfer = 'Transfer #%d has been updated to use the correct currencies. Please verify this in Firefly III.';
$driver = DB::connection()->getDriverName();
$pgsql = ['pgsql', 'postgresql'];
foreach ($types as $type => $operator) {
$query = TransactionJournal
@ -328,10 +329,10 @@ class UpgradeDatabase extends Command
->leftJoin('account_meta', 'account_meta.account_id', '=', 'accounts.id')
->where('transaction_types.type', $type)
->where('account_meta.name', 'currency_id');
if ($driver === 'postgresql') {
if (in_array($driver, $pgsql)) {
$query->where('transaction_journals.transaction_currency_id', '!=', DB::raw('cast(account_meta.data as int)'));
}
if ($driver !== 'postgresql') {
if (!in_array($driver, $pgsql)) {
$query->where('transaction_journals.transaction_currency_id', '!=', DB::raw('account_meta.data'));
}

View File

@ -14,6 +14,7 @@ namespace FireflyIII\Generator\Report\Tag;
use Carbon\Carbon;
use FireflyIII\Generator\Report\ReportGeneratorInterface;
use FireflyIII\Generator\Report\Support;
use FireflyIII\Helpers\Collector\JournalCollectorInterface;
use FireflyIII\Helpers\Filter\NegativeAmountFilter;
use FireflyIII\Helpers\Filter\OpposingAccountFilter;
@ -30,7 +31,7 @@ use Log;
*
* @package FireflyIII\Generator\Report\Tag
*/
class MonthReportGenerator implements ReportGeneratorInterface
class MonthReportGenerator extends Support implements ReportGeneratorInterface
{
/** @var Collection */

View File

@ -18,6 +18,7 @@ use FireflyIII\Models\Attachment;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Support\Collection;
use Illuminate\Support\MessageBag;
use Log;
use Storage;
use Symfony\Component\HttpFoundation\File\UploadedFile;
@ -157,11 +158,14 @@ class AttachmentHelper implements AttachmentHelperInterface
$attachment->size = $file->getSize();
$attachment->uploaded = 0;
$attachment->save();
Log::debug('Created attachment:', $attachment->toArray());
$fileObject = $file->openFile('r');
$fileObject->rewind();
$content = $fileObject->fread($file->getSize());
$encrypted = Crypt::encrypt($content);
Log::debug(sprintf('Full file length is %d and upload size is %d.', strlen($content), $file->getSize()));
Log::debug(sprintf('Encrypted content is %d', strlen($encrypted)));
// store it:
$this->uploadDisk->put($attachment->fileName(), $encrypted);

View File

@ -98,10 +98,14 @@ class AttachmentController extends Controller
*/
public function download(AttachmentRepositoryInterface $repository, Attachment $attachment)
{
if ($repository->exists($attachment)) {
$content = $repository->getContent($attachment);
$quoted = sprintf('"%s"', addcslashes(basename($attachment->filename), '"\\'));
/** @var LaravelResponse $response */
$response = response($content, 200);
$response
@ -149,6 +153,7 @@ class AttachmentController extends Controller
{
$image = 'images/page_green.png';
if ($attachment->mime == 'application/pdf') {
$image = 'images/page_white_acrobat.png';
}

View File

@ -127,6 +127,7 @@ class AccountController extends Controller
$chartData[$account->name] = $diff;
}
}
arsort($chartData);
$data = $this->generator->singleSet(strval(trans('firefly.spent')), $chartData);
$cache->store($data);
@ -424,7 +425,7 @@ class AccountController extends Controller
}
arsort($chartData);
$data = $this->generator->singleSet(strval(trans('firefly.spent')), $chartData);
$data = $this->generator->singleSet(strval(trans('firefly.earned')), $chartData);
$cache->store($data);
return Response::json($data);

View File

@ -12,32 +12,29 @@ declare(strict_types=1);
namespace FireflyIII\Http\Controllers;
use Crypt;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Http\Requests\ImportUploadRequest;
use FireflyIII\Import\ImportProcedureInterface;
use FireflyIII\Import\Setup\SetupInterface;
use FireflyIII\Import\Configurator\ConfiguratorInterface;
use FireflyIII\Import\Routine\ImportRoutine;
use FireflyIII\Models\ImportJob;
use FireflyIII\Repositories\ImportJob\ImportJobRepositoryInterface;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
use FireflyIII\Repositories\User\UserRepositoryInterface;
use Illuminate\Http\Request;
use Illuminate\Http\Response as LaravelResponse;
use Log;
use Response;
use Session;
use SplFileObject;
use Storage;
use Symfony\Component\HttpFoundation\File\UploadedFile;
use View;
/**
* Class ImportController
* Class ImportController.
*
* @package FireflyIII\Http\Controllers
*/
class ImportController extends Controller
{
/** @var ImportJobRepositoryInterface */
public $repository;
/**
*
*/
@ -48,7 +45,8 @@ class ImportController extends Controller
$this->middleware(
function ($request, $next) {
View::share('mainTitleIcon', 'fa-archive');
View::share('title', trans('firefly.import_data_full'));
View::share('title', trans('firefly.import_index_title'));
$this->repository = app(ImportJobRepositoryInterface::class);
return $next($request);
}
@ -56,28 +54,7 @@ class ImportController extends Controller
}
/**
* This is the last step before the import starts.
*
* @param ImportJob $job
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector|View
*/
public function complete(ImportJob $job)
{
Log::debug('Now in complete()', ['job' => $job->key]);
if (!$this->jobInCorrectStep($job, 'complete')) {
return $this->redirectToCorrectStep($job);
}
$subTitle = trans('firefly.import_complete');
$subTitleIcon = 'fa-star';
return view('import.complete', compact('job', 'subTitle', 'subTitleIcon'));
}
/**
* This is step 3.
* This is the first step in configuring the job. It can only be executed
* when the job is set to "import_status_never_started".
* This is step 3. This repeats until the job is configured.
*
* @param ImportJob $job
*
@ -86,35 +63,41 @@ class ImportController extends Controller
*/
public function configure(ImportJob $job)
{
Log::debug('Now at start of configure()');
if (!$this->jobInCorrectStep($job, 'configure')) {
return $this->redirectToCorrectStep($job);
}
// create configuration class:
$configurator = $this->makeConfigurator($job);
// actual code
$importer = $this->makeImporter($job);
$importer->configure();
$data = $importer->getConfigurationData();
$subTitle = trans('firefly.configure_import');
// is the job already configured?
if ($configurator->isJobConfigured()) {
$this->repository->updateStatus($job, 'configured');
return redirect(route('import.status', [$job->key]));
}
$view = $configurator->getNextView();
$data = $configurator->getNextData();
$subTitle = trans('firefly.import_config_bread_crumb');
$subTitleIcon = 'fa-wrench';
return view('import.' . $job->file_type . '.configure', compact('data', 'job', 'subTitle', 'subTitleIcon'));
return view($view, compact('data', 'job', 'subTitle', 'subTitleIcon'));
}
/**
* Generate a JSON file of the job's config and send it to the user.
* Generate a JSON file of the job's configuration and send it to the user.
*
* @param ImportJob $job
*
* @return mixed
* @return LaravelResponse
*/
public function download(ImportJob $job)
{
Log::debug('Now in download()', ['job' => $job->key]);
$config = $job->configuration;
// TODO this is CSV import specific:
$config['column-roles-complete'] = false;
$config['column-mapping-complete'] = false;
$config['initial-config-complete'] = false;
$config['delimiter'] = $config['delimiter'] === "\t" ? 'tab' : $config['delimiter'];
$result = json_encode($config, JSON_PRETTY_PRINT);
$name = sprintf('"%s"', addcslashes('import-configuration-' . date('Y-m-d') . '.json', '"\\'));
@ -134,26 +117,6 @@ class ImportController extends Controller
}
/**
* @param ImportJob $job
*
* @return View
*/
public function finished(ImportJob $job)
{
if (!$this->jobInCorrectStep($job, 'finished')) {
return $this->redirectToCorrectStep($job);
}
// if there is a tag (there might not be), we can link to it:
$tagId = $job->extended_status['importTag'] ?? 0;
$subTitle = trans('firefly.import_finished');
$subTitleIcon = 'fa-star';
return view('import.finished', compact('job', 'subTitle', 'subTitleIcon', 'tagId'));
}
/**
* This is step 1. Upload a file.
*
@ -161,8 +124,7 @@ class ImportController extends Controller
*/
public function index()
{
Log::debug('Now at index');
$subTitle = trans('firefly.import_data_index');
$subTitle = trans('firefly.import_index_sub_title');
$subTitleIcon = 'fa-home';
$importFileTypes = [];
$defaultImportType = config('firefly.default_import_format');
@ -175,6 +137,38 @@ class ImportController extends Controller
}
/**
* This is step 2. It creates an Import Job. Stores the import.
*
* @param ImportUploadRequest $request
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector
*/
public function initialize(ImportUploadRequest $request)
{
Log::debug('Now in initialize()');
// create import job:
$type = $request->get('import_file_type');
$job = $this->repository->create($type);
Log::debug('Created new job', ['key' => $job->key, 'id' => $job->id]);
// process file:
$this->repository->processFile($job, $request->files->get('import_file'));
// process config, if present:
if ($request->files->has('configuration_file')) {
$this->repository->processConfiguration($job, $request->files->get('configuration_file'));
}
$this->repository->updateStatus($job, 'initialized');
return redirect(route('import.configure', [$job->key]));
}
/**
*
* Show status of import job in JSON.
*
* @param ImportJob $job
*
* @return \Illuminate\Http\JsonResponse
@ -182,35 +176,36 @@ class ImportController extends Controller
public function json(ImportJob $job)
{
$result = [
'showPercentage' => false,
'started' => false,
'finished' => false,
'running' => false,
'errors' => $job->extended_status['errors'],
'errors' => array_values($job->extended_status['errors']),
'percentage' => 0,
'steps' => $job->extended_status['total_steps'],
'stepsDone' => $job->extended_status['steps_done'],
'statusText' => trans('firefly.import_status_' . $job->status),
'show_percentage' => false,
'steps' => $job->extended_status['steps'],
'done' => $job->extended_status['done'],
'statusText' => trans('firefly.import_status_job_' . $job->status),
'status' => $job->status,
'finishedText' => '',
];
$percentage = 0;
if ($job->extended_status['total_steps'] !== 0) {
$percentage = round(($job->extended_status['steps_done'] / $job->extended_status['total_steps']) * 100, 0);
if ($job->extended_status['steps'] !== 0) {
$result['percentage'] = round(($job->extended_status['done'] / $job->extended_status['steps']) * 100, 0);
$result['show_percentage'] = true;
}
if ($job->status === 'import_complete') {
$tagId = $job->extended_status['importTag'];
if ($job->status === 'finished') {
$tagId = $job->extended_status['tag'];
/** @var TagRepositoryInterface $repository */
$repository = app(TagRepositoryInterface::class);
$tag = $repository->find($tagId);
$result['finished'] = true;
$result['finishedText'] = trans('firefly.import_finished_link', ['link' => route('tags.show', [$tag->id]), 'tag' => $tag->tag]);
$result['finishedText'] = trans('firefly.import_status_finished_job', ['link' => route('tags.show', [$tag->id, 'all']), 'tag' => $tag->tag]);
}
if ($job->status === 'import_running') {
if ($job->status === 'running') {
$result['started'] = true;
$result['running'] = true;
$result['showPercentage'] = true;
$result['percentage'] = $percentage;
}
return Response::json($result);
@ -220,285 +215,81 @@ class ImportController extends Controller
* Step 4. Save the configuration.
*
* @param Request $request
* @param ImportJobRepositoryInterface $repository
* @param ImportJob $job
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector
*/
public function postConfigure(Request $request, ImportJobRepositoryInterface $repository, ImportJob $job)
public function postConfigure(Request $request, ImportJob $job)
{
Log::debug('Now in postConfigure()', ['job' => $job->key]);
if (!$this->jobInCorrectStep($job, 'process')) {
return $this->redirectToCorrectStep($job);
$configurator = $this->makeConfigurator($job);
// is the job already configured?
if ($configurator->isJobConfigured()) {
return redirect(route('import.status', [$job->key]));
}
Log::debug('Continue postConfigure()', ['job' => $job->key]);
// actual code
$importer = $this->makeImporter($job);
$data = $request->all();
$files = $request->files;
$importer->saveImportConfiguration($data, $files);
$configurator->configureJob($data);
// update job:
$repository->updateStatus($job, 'import_configuration_saved');
// return redirect to settings.
// this could loop until the user is done.
return redirect(route('import.settings', [$job->key]));
// return to configure
return redirect(route('import.configure', [$job->key]));
}
/**
* This step 6. Depending on the importer, this will process the
* settings given and store them.
*
* @param Request $request
* @param ImportJob $job
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector
* @return \Illuminate\Http\JsonResponse
* @throws FireflyException
*/
public function postSettings(Request $request, ImportJob $job)
public function start(ImportJob $job)
{
Log::debug('Now in postSettings()', ['job' => $job->key]);
if (!$this->jobInCorrectStep($job, 'store-settings')) {
return $this->redirectToCorrectStep($job);
/** @var ImportRoutine $routine */
$routine = app(ImportRoutine::class);
$routine->setJob($job);
$result = $routine->run();
if ($result) {
return Response::json(['run' => 'ok']);
}
$importer = $this->makeImporter($job);
$importer->storeSettings($request);
// return redirect to settings (for more settings perhaps)
return redirect(route('import.settings', [$job->key]));
throw new FireflyException('Job did not complete succesfully.');
}
/**
* Step 5. Depending on the importer, this will show the user settings to
* fill in.
*
* @param ImportJobRepositoryInterface $repository
* @param ImportJob $job
*
* @return View
*/
public function settings(ImportJobRepositoryInterface $repository, ImportJob $job)
{
Log::debug('Now in settings()', ['job' => $job->key]);
if (!$this->jobInCorrectStep($job, 'settings')) {
return $this->redirectToCorrectStep($job);
}
Log::debug('Continue in settings()');
$importer = $this->makeImporter($job);
$subTitle = trans('firefly.settings_for_import');
$subTitleIcon = 'fa-wrench';
// now show settings screen to user.
if ($importer->requireUserSettings()) {
Log::debug('Job requires user config.');
$data = $importer->getDataForSettings();
$view = $importer->getViewForSettings();
return view($view, compact('data', 'job', 'subTitle', 'subTitleIcon'));
}
Log::debug('Job does NOT require user config.');
$repository->updateStatus($job, 'settings_complete');
// if no more settings, save job and continue to process thing.
return redirect(route('import.complete', [$job->key]));
// ask the importer for the requested action.
// for example pick columns or map data.
// depends of course on the data in the job.
}
/**
* @param ImportProcedureInterface $importProcedure
* @param ImportJob $job
*/
public function start(ImportProcedureInterface $importProcedure, ImportJob $job)
{
set_time_limit(0);
if ($job->status == 'settings_complete') {
$importProcedure->runImport($job);
}
}
/**
* This is the last step before the import starts.
*
* @param ImportJob $job
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector|View
*/
public function status(ImportJob $job)
{ //
Log::debug('Now in status()', ['job' => $job->key]);
if (!$this->jobInCorrectStep($job, 'status')) {
return $this->redirectToCorrectStep($job);
{
$statuses = ['configured', 'running', 'finished'];
if (!in_array($job->status, $statuses)) {
return redirect(route('import.configure', [$job->key]));
}
$subTitle = trans('firefly.import_status');
$subTitle = trans('firefly.import_status_sub_title');
$subTitleIcon = 'fa-star';
return view('import.status', compact('job', 'subTitle', 'subTitleIcon'));
}
/**
* This is step 2. It creates an Import Job. Stores the import.
*
* @param ImportUploadRequest $request
* @param ImportJobRepositoryInterface $repository
* @param UserRepositoryInterface $userRepository
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector
*/
public function upload(ImportUploadRequest $request, ImportJobRepositoryInterface $repository, UserRepositoryInterface $userRepository)
{
Log::debug('Now in upload()');
// create import job:
$type = $request->get('import_file_type');
$job = $repository->create($type);
Log::debug('Created new job', ['key' => $job->key, 'id' => $job->id]);
/** @var UploadedFile $upload */
$upload = $request->files->get('import_file');
$newName = $job->key . '.upload';
$uploaded = new SplFileObject($upload->getRealPath());
$content = $uploaded->fread($uploaded->getSize());
$contentEncrypted = Crypt::encrypt($content);
$disk = Storage::disk('upload');
// user is demo user, replace upload with prepared file.
if ($userRepository->hasRole(auth()->user(), 'demo')) {
$stubsDisk = Storage::disk('stubs');
$content = $stubsDisk->get('demo-import.csv');
$contentEncrypted = Crypt::encrypt($content);
$disk->put($newName, $contentEncrypted);
Log::debug('Replaced upload with demo file.');
// also set up prepared configuration.
$configuration = json_decode($stubsDisk->get('demo-configuration.json'), true);
$repository->setConfiguration($job, $configuration);
Log::debug('Set configuration for demo user', $configuration);
// also flash info
Session::flash('info', trans('demo.import-configure-security'));
}
if (!$userRepository->hasRole(auth()->user(), 'demo')) {
// user is not demo, process original upload:
$disk->put($newName, $contentEncrypted);
Log::debug('Uploaded file', ['name' => $upload->getClientOriginalName(), 'size' => $upload->getSize(), 'mime' => $upload->getClientMimeType()]);
}
// store configuration file's content into the job's configuration thing. Otherwise, leave it empty.
// demo user's configuration upload is ignored completely.
if ($request->files->has('configuration_file') && !auth()->user()->hasRole('demo')) {
/** @var UploadedFile $configFile */
$configFile = $request->files->get('configuration_file');
Log::debug(
'Uploaded configuration file',
['name' => $configFile->getClientOriginalName(), 'size' => $configFile->getSize(), 'mime' => $configFile->getClientMimeType()]
);
$configFileObject = new SplFileObject($configFile->getRealPath());
$configRaw = $configFileObject->fread($configFileObject->getSize());
$configuration = json_decode($configRaw, true);
// @codeCoverageIgnoreStart
if (!is_null($configuration) && is_array($configuration)) {
Log::debug('Found configuration', $configuration);
$repository->setConfiguration($job, $configuration);
}
// @codeCoverageIgnoreEnd
}
return redirect(route('import.configure', [$job->key]));
}
/**
* @param ImportJob $job
* @param string $method
*
* @return bool
*/
private function jobInCorrectStep(ImportJob $job, string $method): bool
{
Log::debug('Now in jobInCorrectStep()', ['job' => $job->key, 'method' => $method]);
switch ($method) {
case 'configure':
case 'process':
return $job->status === 'import_status_never_started';
case 'settings':
case 'store-settings':
Log::debug(sprintf('Job %d with key %s has status %s', $job->id, $job->key, $job->status));
return $job->status === 'import_configuration_saved';
case 'finished':
return $job->status === 'import_complete';
case 'complete':
return $job->status === 'settings_complete';
case 'status':
return ($job->status === 'settings_complete') || ($job->status === 'import_running');
}
return false; // @codeCoverageIgnore
}
/**
* @param ImportJob $job
*
* @return SetupInterface
* @return ConfiguratorInterface
* @throws FireflyException
*/
private function makeImporter(ImportJob $job): SetupInterface
private function makeConfigurator(ImportJob $job): ConfiguratorInterface
{
// create proper importer (depends on job)
$type = strtolower($job->file_type);
// validate type:
$validTypes = array_keys(config('firefly.import_formats'));
if (in_array($type, $validTypes)) {
/** @var SetupInterface $importer */
$importer = app('FireflyIII\Import\Setup\\' . ucfirst($type) . 'Setup');
$importer->setJob($job);
return $importer;
$type = $job->file_type;
$key = sprintf('firefly.import_configurators.%s', $type);
$className = config($key);
if (is_null($className)) {
throw new FireflyException('Cannot find configurator class for this job.'); // @codeCoverageIgnore
}
throw new FireflyException(sprintf('"%s" is not a valid file type', $type)); // @codeCoverageIgnore
/** @var ConfiguratorInterface $configurator */
$configurator = app($className);
$configurator->setJob($job);
}
/**
* @param ImportJob $job
*
* @return \Illuminate\Http\RedirectResponse|\Illuminate\Routing\Redirector
* @throws FireflyException
*/
private function redirectToCorrectStep(ImportJob $job)
{
Log::debug('Now in redirectToCorrectStep()', ['job' => $job->key]);
switch ($job->status) {
case 'import_status_never_started':
Log::debug('Will redirect to configure()');
return redirect(route('import.configure', [$job->key]));
case 'import_configuration_saved':
Log::debug('Will redirect to settings()');
return redirect(route('import.settings', [$job->key]));
case 'settings_complete':
Log::debug('Will redirect to complete()');
return redirect(route('import.complete', [$job->key]));
case 'import_complete':
Log::debug('Will redirect to finished()');
return redirect(route('import.finished', [$job->key]));
}
throw new FireflyException('Cannot redirect for job state ' . $job->status); // @codeCoverageIgnore
return $configurator;
}
}

View File

@ -120,7 +120,7 @@ class CategoryController extends Controller
foreach ($categories as $category) {
$spent = $repository->spentInPeriod(new Collection([$category]), $accounts, $start, $end);
if (bccomp($spent, '0') !== 0) {
$report[$category->id] = ['name' => $category->name, 'spent' => $spent];
$report[$category->id] = ['name' => $category->name, 'spent' => $spent,'id' => $category->id];
}
}

View File

@ -15,6 +15,7 @@ namespace FireflyIII\Http\Controllers;
use Carbon\Carbon;
use FireflyIII\Helpers\Collector\JournalCollectorInterface;
use FireflyIII\Helpers\Filter\InternalTransferFilter;
use FireflyIII\Http\Requests\TagFormRequest;
use FireflyIII\Models\Tag;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
@ -245,6 +246,7 @@ class TagController extends Controller
$periods = new Collection;
$apiKey = env('GOOGLE_MAPS_API_KEY', '');
$sum = '0';
$path = 'tags/show/' . $tag->id;
// prep for "all" view.
@ -253,6 +255,7 @@ class TagController extends Controller
$start = $repository->firstUseDate($tag);
$end = new Carbon;
$sum = $repository->sumOfTag($tag);
$path = 'tags/show/' . $tag->id . '/all';
}
// prep for "specific date" view.
@ -282,13 +285,13 @@ class TagController extends Controller
Log::info('Now at tag loop start.');
while ($count === 0 && $loop < 3) {
$loop++;
Log::info('Count is zero, search for journals.');
Log::info(sprintf('Count is zero, search for journals between %s and %s (pagesize %d, page %d).', $start, $end, $pageSize, $page));
/** @var JournalCollectorInterface $collector */
$collector = app(JournalCollectorInterface::class);
$collector->setAllAssetAccounts()->setRange($start, $end)->setLimit($pageSize)->setPage($page)->withOpposingAccount()
->setTag($tag)->withBudgetInformation()->withCategoryInformation();
->setTag($tag)->withBudgetInformation()->withCategoryInformation()->removeFilter(InternalTransferFilter::class);
$journals = $collector->getPaginatedJournals();
$journals->setPath('tags/show/' . $tag->id);
$journals->setPath($path);
$count = $journals->getCollection()->count();
if ($count === 0) {
$start->subDay();

View File

@ -229,8 +229,8 @@ class ConvertController extends Controller
switch ($joined) {
default:
throw new FireflyException('Cannot handle ' . $joined); // @codeCoverageIgnore
case TransactionType::WITHDRAWAL . '-' . TransactionType::DEPOSIT: // one
case TransactionType::TRANSFER . '-' . TransactionType::DEPOSIT: // six
case TransactionType::WITHDRAWAL . '-' . TransactionType::DEPOSIT:
case TransactionType::TRANSFER . '-' . TransactionType::DEPOSIT:
if ($data['source_account_revenue'] === '') {
// destination is a cash account.
@ -248,14 +248,14 @@ class ConvertController extends Controller
];
$source = $accountRepository->store($data);
break;
case TransactionType::WITHDRAWAL . '-' . TransactionType::TRANSFER: // two
case TransactionType::TRANSFER . '-' . TransactionType::WITHDRAWAL: // five
case TransactionType::WITHDRAWAL . '-' . TransactionType::TRANSFER:
case TransactionType::TRANSFER . '-' . TransactionType::WITHDRAWAL:
$source = $sourceAccount;
break;
case TransactionType::DEPOSIT . '-' . TransactionType::WITHDRAWAL: // three
case TransactionType::DEPOSIT . '-' . TransactionType::WITHDRAWAL:
$source = $destinationAccount;
break;
case TransactionType::DEPOSIT . '-' . TransactionType::TRANSFER: // four
case TransactionType::DEPOSIT . '-' . TransactionType::TRANSFER:
$source = $accountRepository->find(intval($data['source_account_asset']));
break;
}

View File

@ -93,7 +93,9 @@ class SingleController extends Controller
$category = $journal->categories()->first();
$categoryName = is_null($category) ? '' : $category->name;
$tags = join(',', $journal->tags()->get()->pluck('tag')->toArray());
$transaction = $journal->transactions()->first();
$amount = Steam::positive($transaction->amount);
$foreignAmount = is_null($transaction->foreign_amount) ? null : Steam::positive($transaction->foreign_amount);
$preFilled = [
'description' => $journal->description,
@ -101,7 +103,10 @@ class SingleController extends Controller
'source_account_name' => $source->name,
'destination_account_id' => $destination->id,
'destination_account_name' => $destination->name,
'amount' => $journal->amountPositive(),
'amount' => $amount,
'source_amount' => $amount,
'destination_amount' => $foreignAmount,
'foreign_amount' => $foreignAmount,
'date' => $journal->date->format('Y-m-d'),
'budget_id' => $budgetId,
'category' => $categoryName,

View File

@ -218,7 +218,7 @@ class SplitController extends Controller
'journal_destination_account_id' => $request->old('journal_destination_account_id', $destinationAccounts->first()->id),
'destinationAccounts' => $destinationAccounts,
'what' => strtolower($journal->transactionTypeStr()),
'date' => $request->old('date', $journal->date),
'date' => $request->old('date', $journal->date->format('Y-m-d')),
'tags' => join(',', $journal->tags->pluck('tag')->toArray()),
// all custom fields:

View File

@ -78,6 +78,7 @@ class TransactionController extends Controller
$start = null;
$end = null;
$periods = new Collection;
$path = '/transactions/' . $what;
// prep for "all" view.
if ($moment === 'all') {
@ -85,6 +86,7 @@ class TransactionController extends Controller
$first = $repository->first();
$start = $first->date ?? new Carbon;
$end = new Carbon;
$path = '/transactions/'.$what.'/all/';
}
// prep for "specific date" view.
@ -118,7 +120,7 @@ class TransactionController extends Controller
$collector->setAllAssetAccounts()->setRange($start, $end)->setTypes($types)->setLimit($pageSize)->setPage($page)->withOpposingAccount();
$collector->removeFilter(InternalTransferFilter::class);
$journals = $collector->getPaginatedJournals();
$journals->setPath('/transactions/' . $what);
$journals->setPath($path);
$count = $journals->getCollection()->count();
if ($count === 0) {
$start->subDay();

View File

@ -40,6 +40,7 @@ class ImportUploadRequest extends Request
return [
'import_file' => 'required|file',
'import_file_type' => 'required|in:' . join(',', $types),
'configuration_file' => 'file',
];
}

View File

@ -469,26 +469,20 @@ Breadcrumbs::register(
$breadcrumbs->push(trans('firefly.import'), route('import.index'));
}
);
Breadcrumbs::register(
'import.complete', function (BreadCrumbGenerator $breadcrumbs, ImportJob $job) {
$breadcrumbs->parent('import.index');
$breadcrumbs->push(trans('firefly.bread_crumb_import_complete', ['key' => $job->key]), route('import.complete', [$job->key]));
}
);
Breadcrumbs::register(
'import.configure', function (BreadCrumbGenerator $breadcrumbs, ImportJob $job) {
$breadcrumbs->parent('import.index');
$breadcrumbs->push(trans('firefly.bread_crumb_configure_import', ['key' => $job->key]), route('import.configure', [$job->key]));
$breadcrumbs->push(trans('firefly.import_config_sub_title', ['key' => $job->key]), route('import.configure', [$job->key]));
}
);
Breadcrumbs::register(
'import.finished', function (BreadCrumbGenerator $breadcrumbs, ImportJob $job) {
'import.status', function (BreadCrumbGenerator $breadcrumbs, ImportJob $job) {
$breadcrumbs->parent('import.index');
$breadcrumbs->push(trans('firefly.bread_crumb_import_finished', ['key' => $job->key]), route('import.finished', [$job->key]));
$breadcrumbs->push(trans('firefly.import_status_bread_crumb', ['key' => $job->key]), route('import.status', [$job->key]));
}
);
/**
* PREFERENCES
*/

View File

@ -0,0 +1,65 @@
<?php
/**
* ConfiguratorInterface.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Configurator;
use FireflyIII\Models\ImportJob;
/**
* Interface ConfiguratorInterface
*
* @package FireflyIII\Import\Configurator
*/
interface ConfiguratorInterface
{
/**
* ConfiguratorInterface constructor.
*/
public function __construct();
/**
* Store any data from the $data array into the job.
*
* @param array $data
*
* @return bool
*/
public function configureJob(array $data): bool;
/**
* @param ImportJob $job
*
* @return void
*/
public function setJob(ImportJob $job);
/**
* Return the data required for the next step in the job configuration.
*
* @return array
*/
public function getNextData(): array;
/**
* Returns the view of the next step in the job configuration.
*
* @return string
*/
public function getNextView(): string;
/**
* Returns true when the initial configuration for this job is complete.
*
* @return bool
*/
public function isJobConfigured(): bool;
}

View File

@ -0,0 +1,159 @@
<?php
/**
* CsvConfigurator.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Configurator;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\ImportJob;
use FireflyIII\Support\Import\Configuration\ConfigurationInterface;
use FireflyIII\Support\Import\Configuration\Csv\Initial;
use FireflyIII\Support\Import\Configuration\Csv\Map;
use FireflyIII\Support\Import\Configuration\Csv\Roles;
use Log;
/**
* Class CsvConfigurator
*
* @package FireflyIII\Import\Configurator
*/
class CsvConfigurator implements ConfiguratorInterface
{
private $job;
/**
* ConfiguratorInterface constructor.
*/
public function __construct()
{
}
/**
* Store any data from the $data array into the job.
*
* @param array $data
*
* @return bool
* @throws FireflyException
*/
public function configureJob(array $data): bool
{
$class = $this->getConfigurationClass();
$job = $this->job;
/** @var ConfigurationInterface $object */
$object = new $class($this->job);
$object->setJob($job);
return $object->storeConfiguration($data);
}
/**
* Return the data required for the next step in the job configuration.
*
* @return array
* @throws FireflyException
*/
public function getNextData(): array
{
$class = $this->getConfigurationClass();
$job = $this->job;
/** @var ConfigurationInterface $object */
$object = app($class);
$object->setJob($job);
return $object->getData();
}
/**
* @return string
* @throws FireflyException
*/
public function getNextView(): string
{
if (!$this->job->configuration['initial-config-complete']) {
return 'import.csv.initial';
}
if (!$this->job->configuration['column-roles-complete']) {
return 'import.csv.roles';
}
if (!$this->job->configuration['column-mapping-complete']) {
return 'import.csv.map';
}
throw new FireflyException('No view for state');
}
/**
* @return bool
*/
public function isJobConfigured(): bool
{
$config = $this->job->configuration;
$config['initial-config-complete'] = $config['initial-config-complete'] ?? false;
$config['column-roles-complete'] = $config['column-roles-complete'] ?? false;
$config['column-mapping-complete'] = $config['column-mapping-complete'] ?? false;
$this->job->configuration = $config;
$this->job->save();
if ($this->job->configuration['initial-config-complete']
&& $this->job->configuration['column-roles-complete']
&& $this->job->configuration['column-mapping-complete']
) {
return true;
}
return false;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
if (is_null($this->job->configuration) || count($this->job->configuration) === 0) {
Log::debug(sprintf('Gave import job %s initial configuration.', $this->job->key));
$this->job->configuration = config('csv.default_config');
$this->job->save();
}
}
/**
* @return string
* @throws FireflyException
*/
private function getConfigurationClass(): string
{
$class = false;
switch (true) {
case (!$this->job->configuration['initial-config-complete']):
$class = Initial::class;
break;
case (!$this->job->configuration['column-roles-complete']):
$class = Roles::class;
break;
case (!$this->job->configuration['column-mapping-complete']):
$class = Map::class;
break;
default:
break;
}
if ($class === false || strlen($class) === 0) {
throw new FireflyException('Cannot handle current job state in getConfigurationClass().');
}
if (!class_exists($class)) {
throw new FireflyException(sprintf('Class %s does not exist in getConfigurationClass().', $class));
}
return $class;
}
}

View File

@ -1,69 +0,0 @@
<?php
/**
* AccountId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class AccountId
*
* @package FireflyIII\Import\Converter
*/
class AccountId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value)
{
$value = intval(trim($value));
Log::debug('Going to convert using AssetAccountId', ['value' => $value]);
if ($value === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
$this->setCertainty(100);
return $account;
}
}
$account = $repository->find($value);// not mapped? Still try to find it first:
if (!is_null($account->id)) {
$this->setCertainty(90);
Log::debug('Found account by ID ', ['id' => $account->id]);
return $account;
}
$this->setCertainty(0); // should not really happen. If the ID does not match FF, what is FF supposed to do?
return new Account;
}
}

View File

@ -18,7 +18,7 @@ namespace FireflyIII\Import\Converter;
*
* @package FireflyIII\Import\Converter
*/
class Amount extends BasicConverter implements ConverterInterface
class Amount implements ConverterInterface
{
/**
@ -28,9 +28,9 @@ class Amount extends BasicConverter implements ConverterInterface
*
* @param $value
*
* @return float
* @return string
*/
public function convert($value): float
public function convert($value): string
{
$len = strlen($value);
$decimalPosition = $len - 3;
@ -59,10 +59,7 @@ class Amount extends BasicConverter implements ConverterInterface
$value = str_replace($search, '', $value);
}
$this->setCertainty(90);
return round(floatval($value), 12);
return strval(round(floatval($value), 12));
}
}

View File

@ -1,87 +0,0 @@
<?php
/**
* AssetAccountIban.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class AssetAccountIban
*
* @package FireflyIII\Import\Converter
*/
class AssetAccountIban extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value): Account
{
$value = trim($value);
Log::debug('Going to convert ', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
$this->setCertainty(100);
Log::debug('Found account by ID', ['id' => $account->id]);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByIban($value, [AccountType::ASSET]);
if (!is_null($account->id)) {
Log::debug('Found account by IBAN', ['id' => $account->id]);
$this->setCertainty(50);
return $account;
}
$account = $repository->store(
['name' => 'Asset account with IBAN ' . $value, 'iban' => $value, 'user' => $this->user->id, 'accountType' => 'asset', 'virtualBalance' => 0,
'active' => true, 'openingBalance' => 0]
);
if (is_null($account->id)) {
$this->setCertainty(0);
Log::info('Could not store new asset account by IBAN', $account->getErrors()->toArray());
return new Account;
}
$this->setCertainty(100);
return $account;
}
}

View File

@ -1,90 +0,0 @@
<?php
/**
* AssetAccountName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class AssetAccountName
*
* @package FireflyIII\Import\Converter
*/
class AssetAccountName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using AssetAccountName', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
$this->setCertainty(100);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByName($value, [AccountType::ASSET]);
if (!is_null($account->id)) {
Log::debug('Found asset account by name', ['value' => $value, 'id' => $account->id]);
return $account;
}
$account = $repository->store(
['name' => $value, 'iban' => null, 'openingBalance' => 0, 'user' => $this->user->id, 'accountType' => 'asset', 'virtualBalance' => 0,
'active' => true]
);
if (is_null($account->id)) {
$this->setCertainty(0);
Log::info('Could not store new asset account by name', $account->getErrors()->toArray());
return new Account;
}
$this->setCertainty(100);
Log::debug('Created new asset account ', ['name' => $account->name, 'id' => $account->id]);
return $account;
}
}

View File

@ -1,96 +0,0 @@
<?php
/**
* AssetAccountNumber.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class AssetAccountNumber
*
* @package FireflyIII\Import\Converter
*/
class AssetAccountNumber extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using AssetAccountNumber', ['value' => $value]);
if (strlen($value) === 0) {
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByAccountNumber($value, [AccountType::ASSET]);
if (!is_null($account->id)) {
Log::debug('Found account by name', ['id' => $account->id]);
$this->setCertainty(50);
return $account;
}
// try to find by the name we would give it:
$accountName = 'Asset account with number ' . e($value);
$account = $repository->findByName($accountName, [AccountType::ASSET]);
if (!is_null($account->id)) {
Log::debug('Found account by name', ['id' => $account->id]);
$this->setCertainty(50);
return $account;
}
$account = $repository->store(
['name' => $accountName, 'openingBalance' => 0, 'iban' => null, 'user' => $this->user->id,
'accountType' => 'asset',
'virtualBalance' => 0, 'accountNumber' => $value, 'active' => true]
);
if (is_null($account->id)) {
$this->setCertainty(0);
Log::info('Could not store new asset account by account number', $account->getErrors()->toArray());
return new Account;
}
$this->setCertainty(100);
return $account;
}
}

View File

@ -1,85 +0,0 @@
<?php
/**
* BasicConverter.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\User;
/**
* Class BasicConverter
*
* @package FireflyIII\Import\Converter
*/
class BasicConverter
{
/** @var int */
public $certainty = 50;
/** @var array */
public $config;
/** @var bool */
public $doMap;
/** @var array */
public $mapping = [];
/** @var User */
public $user;
/**
* @return int
*/
public function getCertainty(): int
{
return $this->certainty;
}
/**
* @param int $certainty
*/
protected function setCertainty(int $certainty)
{
$this->certainty = $certainty;
}
/**
* @param array $config
*/
public function setConfig(array $config)
{
$this->config = $config;
}
/**
* @param mixed $doMap
*/
public function setDoMap(bool $doMap)
{
$this->doMap = $doMap;
}
/**
* @param array $mapping
*
*/
public function setMapping(array $mapping)
{
$this->mapping = $mapping;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
}
}

View File

@ -1,76 +0,0 @@
<?php
/**
* BillId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Bill;
use FireflyIII\Repositories\Bill\BillRepositoryInterface;
use Log;
/**
* Class BillId
*
* @package FireflyIII\Import\Converter
*/
class BillId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Bill
*/
public function convert($value)
{
$value = intval(trim($value));
Log::debug('Going to convert using BillId', ['value' => $value]);
if ($value === 0) {
$this->setCertainty(0);
return new Bill;
}
/** @var BillRepositoryInterface $repository */
$repository = app(BillRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found bill in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$bill = $repository->find(intval($this->mapping[$value]));
if (!is_null($bill->id)) {
Log::debug('Found bill by ID', ['id' => $bill->id]);
$this->setCertainty(100);
return $bill;
}
}
// not mapped? Still try to find it first:
$bill = $repository->find($value);
if (!is_null($bill->id)) {
Log::debug('Found bill by ID ', ['id' => $bill->id]);
$this->setCertainty(100);
return $bill;
}
// should not really happen. If the ID does not match FF, what is FF supposed to do?
Log::info(sprintf('Could not find bill with ID %d. Will return NULL', $value));
$this->setCertainty(0);
return new Bill;
}
}

View File

@ -1,99 +0,0 @@
<?php
/**
* BillName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\Bill;
use FireflyIII\Repositories\Bill\BillRepositoryInterface;
use Log;
/**
* Class BillName
*
* @package FireflyIII\Import\Converter
*/
class BillName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Bill
* @throws FireflyException
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using BillName', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Bill;
}
/** @var BillRepositoryInterface $repository */
$repository = app(BillRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found bill in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$bill = $repository->find(intval($this->mapping[$value]));
if (!is_null($bill->id)) {
Log::debug('Found bill by ID', ['id' => $bill->id]);
$this->setCertainty(100);
return $bill;
}
}
// not mapped? Still try to find it first:
$bill = $repository->findByName($value);
if (!is_null($bill->id)) {
Log::debug('Found bill by name ', ['id' => $bill->id]);
$this->setCertainty(100);
return $bill;
}
// create new bill. Use a lot of made up values.
$bill = $repository->store(
[
'name' => $value,
'match' => $value,
'amount_min' => 1,
'user' => $this->user->id,
'amount_max' => 10,
'date' => date('Ymd'),
'repeat_freq' => 'monthly',
'skip' => 0,
'automatch' => 0,
'active' => 1,
]
);
if (is_null($bill->id)) {
$this->setCertainty(0);
Log::info('Could not store new bill by name', $bill->getErrors()->toArray());
return new Bill;
}
$this->setCertainty(100);
return $bill;
}
}

View File

@ -1,76 +0,0 @@
<?php
/**
* BudgetId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Budget;
use FireflyIII\Repositories\Budget\BudgetRepositoryInterface;
use Log;
/**
* Class BudgetId
*
* @package FireflyIII\Import\Converter
*/
class BudgetId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Budget
*/
public function convert($value)
{
$value = intval(trim($value));
Log::debug('Going to convert using BudgetId', ['value' => $value]);
if ($value === 0) {
$this->setCertainty(0);
return new Budget;
}
/** @var BudgetRepositoryInterface $repository */
$repository = app(BudgetRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found budget in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$budget = $repository->find(intval($this->mapping[$value]));
if (!is_null($budget->id)) {
Log::debug('Found budget by ID', ['id' => $budget->id]);
$this->setCertainty(100);
return $budget;
}
}
// not mapped? Still try to find it first:
$budget = $repository->find($value);
if (!is_null($budget->id)) {
Log::debug('Found budget by ID ', ['id' => $budget->id]);
$this->setCertainty(100);
return $budget;
}
// should not really happen. If the ID does not match FF, what is FF supposed to do?
$this->setCertainty(0);
Log::info(sprintf('Could not find budget with ID %d. Will return NULL', $value));
return new Budget;
}
}

View File

@ -1,80 +0,0 @@
<?php
/**
* BudgetName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Budget;
use FireflyIII\Repositories\Budget\BudgetRepositoryInterface;
use Log;
/**
* Class BudgetName
*
* @package FireflyIII\Import\Converter
*/
class BudgetName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Budget
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using BudgetName', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Budget;
}
/** @var BudgetRepositoryInterface $repository */
$repository = app(BudgetRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found budget in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$budget = $repository->find(intval($this->mapping[$value]));
if (!is_null($budget->id)) {
Log::debug('Found budget by ID', ['id' => $budget->id]);
$this->setCertainty(100);
return $budget;
}
}
// not mapped? Still try to find it first:
$budget = $repository->findByName($value);
if (!is_null($budget->id)) {
Log::debug('Found budget by name ', ['id' => $budget->id]);
$this->setCertainty(100);
return $budget;
}
// create new budget. Use a lot of made up values.
$budget = $repository->store(
[
'name' => $value,
'user' => $this->user->id,
]
);
$this->setCertainty(100);
return $budget;
}
}

View File

@ -1,76 +0,0 @@
<?php
/**
* CategoryId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Category;
use FireflyIII\Repositories\Category\CategoryRepositoryInterface;
use Log;
/**
* Class CategoryId
*
* @package FireflyIII\Import\Converter
*/
class CategoryId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Category
*/
public function convert($value)
{
$value = intval(trim($value));
Log::debug('Going to convert using CategoryId', ['value' => $value]);
if ($value === 0) {
$this->setCertainty(0);
return new Category;
}
/** @var CategoryRepositoryInterface $repository */
$repository = app(CategoryRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found category in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$category = $repository->find(intval($this->mapping[$value]));
if (!is_null($category->id)) {
Log::debug('Found category by ID', ['id' => $category->id]);
$this->setCertainty(100);
return $category;
}
}
// not mapped? Still try to find it first:
$category = $repository->find($value);
if (!is_null($category->id)) {
Log::debug('Found category by ID ', ['id' => $category->id]);
$this->setCertainty(100);
return $category;
}
// should not really happen. If the ID does not match FF, what is FF supposed to do?
$this->setCertainty(0);
Log::info(sprintf('Could not find category with ID %d. Will return NULL', $value));
return new Category;
}
}

View File

@ -1,80 +0,0 @@
<?php
/**
* CategoryName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Category;
use FireflyIII\Repositories\Category\CategoryRepositoryInterface;
use Log;
/**
* Class CategoryName
*
* @package FireflyIII\Import\Converter
*/
class CategoryName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Category
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using CategoryName', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Category;
}
/** @var CategoryRepositoryInterface $repository */
$repository = app(CategoryRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found category in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$category = $repository->find(intval($this->mapping[$value]));
if (!is_null($category->id)) {
Log::debug('Found category by ID', ['id' => $category->id]);
$this->setCertainty(100);
return $category;
}
}
// not mapped? Still try to find it first:
$category = $repository->findByName($value);
if (!is_null($category->id)) {
Log::debug('Found category by name ', ['id' => $category->id]);
$this->setCertainty(100);
return $category;
}
// create new category. Use a lot of made up values.
$category = $repository->store(
[
'name' => $value,
'user' => $this->user->id,
]
);
$this->setCertainty(100);
return $category;
}
}

View File

@ -13,8 +13,6 @@ declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\User;
/**
* Interface ConverterInterface
*
@ -27,30 +25,4 @@ interface ConverterInterface
*
*/
public function convert($value);
/**
* @return int
*/
public function getCertainty(): int;
/**
* @param array $config
*/
public function setConfig(array $config);
/**
* @param bool $doMap
*/
public function setDoMap(bool $doMap);
/**
* @param array $mapping
*
*/
public function setMapping(array $mapping);
/**
* @param User $user
*/
public function setUser(User $user);
}

View File

@ -1,71 +0,0 @@
<?php
/**
* CurrencyCode.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use Log;
/**
* Class CurrencyCode
*
* @package FireflyIII\Import\Converter
*/
class CurrencyCode extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return TransactionCurrency
*/
public function convert($value): TransactionCurrency
{
Log::debug('Going to convert currency code', ['value' => $value]);
/** @var CurrencyRepositoryInterface $repository */
$repository = app(CurrencyRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found currency in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$currency = $repository->find(intval($this->mapping[$value]));
if (!is_null($currency->id)) {
Log::debug('Found currency by ID', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
}
// not mapped? Still try to find it first:
$currency = $repository->findByCode($value);
if (!is_null($currency->id)) {
Log::debug('Found currency by code', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
$currency = $repository->store(
[
'name' => $value,
'code' => $value,
'symbol' => $value,
]
);
$this->setCertainty(100);
return $currency;
}
}

View File

@ -1,75 +0,0 @@
<?php
/**
* CurrencyId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use Log;
/**
* Class CurrencyId
*
* @package FireflyIII\Import\Converter
*/
class CurrencyId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return TransactionCurrency
*/
public function convert($value)
{
$value = intval(trim($value));
Log::debug('Going to convert using CurrencyId', ['value' => $value]);
if ($value === 0) {
$this->setCertainty(0);
return new TransactionCurrency;
}
/** @var CurrencyRepositoryInterface $repository */
$repository = app(CurrencyRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found currency in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$currency = $repository->find(intval($this->mapping[$value]));
if (!is_null($currency->id)) {
Log::debug('Found currency by ID', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
}
// not mapped? Still try to find it first:
$currency = $repository->find($value);
if (!is_null($currency->id)) {
Log::debug('Found currency by ID ', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
$this->setCertainty(0);
// should not really happen. If the ID does not match FF, what is FF supposed to do?
Log::info(sprintf('Could not find category with ID %d. Will return NULL', $value));
return new TransactionCurrency;
}
}

View File

@ -1,81 +0,0 @@
<?php
/**
* CurrencyName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use Log;
/**
* Class CurrencyName
*
* @package FireflyIII\Import\Converter
*/
class CurrencyName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return TransactionCurrency
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using CurrencyName', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new TransactionCurrency;
}
/** @var CurrencyRepositoryInterface $repository */
$repository = app(CurrencyRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found currency in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$currency = $repository->find(intval($this->mapping[$value]));
if (!is_null($currency->id)) {
Log::debug('Found currency by ID', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
}
// not mapped? Still try to find it first:
$currency = $repository->findByName($value);
if (!is_null($currency->id)) {
Log::debug('Found currency by name ', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
// create new currency
$currency = $repository->store(
[
'name' => $value,
'code' => strtoupper(substr($value, 0, 3)),
'symbol' => strtoupper(substr($value, 0, 1)),
]
);
$this->setCertainty(100);
return $currency;
}
}

View File

@ -1,81 +0,0 @@
<?php
/**
* CurrencySymbol.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use Log;
/**
* Class CurrencySymbol
*
* @package FireflyIII\Import\Converter
*/
class CurrencySymbol extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return TransactionCurrency
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using CurrencySymbol', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new TransactionCurrency;
}
/** @var CurrencyRepositoryInterface $repository */
$repository = app(CurrencyRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found currency in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$currency = $repository->find(intval($this->mapping[$value]));
if (!is_null($currency->id)) {
Log::debug('Found currency by ID', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
}
// not mapped? Still try to find it first:
$currency = $repository->findBySymbol($value);
if (!is_null($currency->id)) {
Log::debug('Found currency by symbol ', ['id' => $currency->id]);
$this->setCertainty(100);
return $currency;
}
// create new currency
$currency = $repository->store(
[
'name' => 'Currency ' . $value,
'code' => $value,
'symbol' => $value,
]
);
$this->setCertainty(100);
return $currency;
}
}

View File

@ -1,53 +0,0 @@
<?php
/**
* Date.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use Carbon\Carbon;
use FireflyIII\Exceptions\FireflyException;
use InvalidArgumentException;
use Log;
/**
* Class Date
*
* @package FireflyIII\Import\Converter
*/
class Date extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Carbon
* @throws FireflyException
*/
public function convert($value): Carbon
{
Log::debug('Going to convert date', ['value' => $value]);
Log::debug('Format: ', ['format' => $this->config['date-format']]);
try {
$date = Carbon::createFromFormat($this->config['date-format'], $value);
} catch (InvalidArgumentException $e) {
Log::info($e->getMessage());
Log::info('Cannot convert this string using the given format.', ['value' => $value, 'format' => $this->config['date-format']]);
$this->setCertainty(0);
return new Carbon;
}
Log::debug('Converted date', ['converted' => $date->toAtomString()]);
$this->setCertainty(100);
return $date;
}
}

View File

@ -1,39 +0,0 @@
<?php
/**
* Description.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
/**
* Class Description
*
* @package FireflyIII\Import\Converter
*/
class Description extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return string
*/
public function convert($value): string
{
// this should replace all control characters
// but leave utf8 intact:
$value = preg_replace('/[\x00-\x1F\x80-\x9F]/u', '', $value);
$this->setCertainty(100);
return strval($value);
}
}

View File

@ -1,39 +0,0 @@
<?php
/**
* ExternalId.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
/**
* Class ExternalId
*
* @package FireflyIII\Import\Converter
*/
class ExternalId extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return string
*/
public function convert($value): string
{
// this should replace all control characters
// but leave utf8 intact:
$value = preg_replace('/[\x00-\x1F\x80-\x9F]/u', '', $value);
$this->setCertainty(100);
return strval(trim($value));
}
}

View File

@ -20,7 +20,7 @@ use Log;
*
* @package FireflyIII\Import\Converter
*/
class INGDebetCredit extends BasicConverter implements ConverterInterface
class INGDebetCredit implements ConverterInterface
{
/**
@ -34,12 +34,10 @@ class INGDebetCredit extends BasicConverter implements ConverterInterface
if ($value === 'Af') {
Log::debug('Return -1');
$this->setCertainty(100);
return -1;
}
$this->setCertainty(100);
Log::debug('Return 1');
return 1;

View File

@ -1,34 +0,0 @@
<?php
/**
* Ignore.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
/**
* Class Ignore
*
* @package FireflyIII\Import\Converter
*/
class Ignore extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return null
*/
public function convert($value)
{
return null;
}
}

View File

@ -1,91 +0,0 @@
<?php
/**
* OpposingAccountIban.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\Account;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class OpposingAccountIban
*
* @package FireflyIII\Import\Converter
*/
class OpposingAccountIban extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value): Account
{
$value = trim($value);
Log::debug('Going to convert opposing IBAN', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
$this->setCertainty(100);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByIban($value, []);
if (!is_null($account->id)) {
Log::debug('Found account by IBAN', ['id' => $account->id]);
Log::info(
'The match between IBAN and account is uncertain because the type of transactions may not have been determined.',
['id' => $account->id, 'iban' => $value]
);
$this->setCertainty(50);
return $account;
}
// the IBAN given may not be a valid IBAN. If not, we cannot store by
// iban and we have no opposing account. There should be some kind of fall back
// routine.
try {
$account = $repository->store(
['name' => $value, 'iban' => $value, 'user' => $this->user->id, 'accountType' => 'import', 'virtualBalance' => 0, 'active' => true,
'openingBalance' => 0]
);
$this->setCertainty(100);
} catch (FireflyException $e) {
Log::error($e);
$account = new Account;
}
return $account;
}
}

View File

@ -1,89 +0,0 @@
<?php
/**
* OpposingAccountName.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class OpposingAccountName
*
* @package FireflyIII\Import\Converter
*/
class OpposingAccountName extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value): Account
{
$value = trim($value);
Log::debug('Going to convert opposing account name', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
$this->setCertainty(100);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByName($value, []);
if (!is_null($account->id)) {
Log::debug('Found opposing account by name', ['id' => $account->id]);
Log::info(
'The match between name and account is uncertain because the type of transactions may not have been determined.',
['id' => $account->id, 'name' => $value]
);
$this->setCertainty(50);
return $account;
}
$account = $repository->store(
['name' => $value, 'iban' => null, 'user' => $this->user->id, 'accountType' => 'import', 'virtualBalance' => 0, 'active' => true,
'openingBalance' => 0,
]
);
if (is_null($account->id)) {
$this->setCertainty(0);
return new Account;
}
$this->setCertainty(100);
Log::debug('Created new opposing account ', ['name' => $account->name, 'id' => $account->id]);
return $account;
}
}

View File

@ -1,91 +0,0 @@
<?php
/**
* OpposingAccountNumber.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Log;
/**
* Class OpposingAccountNumber
*
* @package FireflyIII\Import\Converter
*/
class OpposingAccountNumber extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Account
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using OpposingAccountNumber', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Account;
}
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
if (isset($this->mapping[$value])) {
Log::debug('Found account in mapping. Should exist.', ['value' => $value, 'map' => $this->mapping[$value]]);
$account = $repository->find(intval($this->mapping[$value]));
if (!is_null($account->id)) {
Log::debug('Found account by ID', ['id' => $account->id]);
$this->setCertainty(100);
return $account;
}
}
// not mapped? Still try to find it first:
$account = $repository->findByAccountNumber($value, []);
if (!is_null($account->id)) {
Log::debug('Found account by number', ['id' => $account->id]);
$this->setCertainty(50);
return $account;
}
// try to find by the name we would give it:
$accountName = 'Import account with number ' . e($value);
$account = $repository->findByName($accountName, [AccountType::IMPORT]);
if (!is_null($account->id)) {
Log::debug('Found account by name', ['id' => $account->id]);
$this->setCertainty(50);
return $account;
}
$account = $repository->store(
['name' => $accountName, 'openingBalance' => 0, 'iban' => null, 'user' => $this->user->id,
'accountType' => 'import',
'virtualBalance' => 0, 'accountNumber' => $value, 'active' => true]
);
$this->setCertainty(100);
return $account;
}
}

View File

@ -20,7 +20,7 @@ use Log;
*
* @package FireflyIII\Import\Converter
*/
class RabobankDebetCredit extends BasicConverter implements ConverterInterface
class RabobankDebetCredit implements ConverterInterface
{
/**
@ -34,13 +34,11 @@ class RabobankDebetCredit extends BasicConverter implements ConverterInterface
if ($value === 'D') {
Log::debug('Return -1');
$this->setCertainty(100);
return -1;
}
Log::debug('Return 1');
$this->setCertainty(100);
return 1;
}

View File

@ -1,86 +0,0 @@
<?php
/**
* TagSplit.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* Class TagSplit
*
* @package FireflyIII\Import\Converter
*/
class TagSplit
{
/**
* @param User $user
* @param array $mapping
* @param array $parts
*
* @return Collection
*/
public static function createSetFromSplits(User $user, array $mapping, array $parts): Collection
{
$set = new Collection;
Log::debug('Exploded parts.', $parts);
/** @var TagRepositoryInterface $repository */
$repository = app(TagRepositoryInterface::class);
$repository->setUser($user);
/** @var string $part */
foreach ($parts as $part) {
if (isset($mapping[$part])) {
Log::debug('Found tag in mapping. Should exist.', ['value' => $part, 'map' => $mapping[$part]]);
$tag = $repository->find(intval($mapping[$part]));
if (!is_null($tag->id)) {
Log::debug('Found tag by ID', ['id' => $tag->id]);
$set->push($tag);
continue;
}
}
// not mapped? Still try to find it first:
$tag = $repository->findByTag($part);
if (!is_null($tag->id)) {
Log::debug('Found tag by name ', ['id' => $tag->id]);
$set->push($tag);
}
if (is_null($tag->id)) {
// create new tag
$tag = $repository->store(
[
'tag' => $part,
'date' => null,
'description' => $part,
'latitude' => null,
'longitude' => null,
'zoomLevel' => null,
'tagMode' => 'nothing',
]
);
Log::debug('Created new tag', ['name' => $part, 'id' => $tag->id]);
$set->push($tag);
}
}
return $set;
}
}

View File

@ -1,48 +0,0 @@
<?php
/**
* TagsComma.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use Illuminate\Support\Collection;
use Log;
/**
* Class TagsComma
*
* @package FireflyIII\Import\Converter
*/
class TagsComma extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Collection
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using TagsComma', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Collection;
}
$parts = array_unique(explode(',', $value));
$set = TagSplit::createSetFromSplits($this->user, $this->mapping, $parts);
$this->setCertainty(100);
return $set;
}
}

View File

@ -1,49 +0,0 @@
<?php
/**
* TagsSpace.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Converter;
use Illuminate\Support\Collection;
use Log;
/**
* Class TagsSpace
*
* @package FireflyIII\Import\Converter
*/
class TagsSpace extends BasicConverter implements ConverterInterface
{
/**
* @param $value
*
* @return Collection
*/
public function convert($value)
{
$value = trim($value);
Log::debug('Going to convert using TagsSpace', ['value' => $value]);
if (strlen($value) === 0) {
$this->setCertainty(0);
return new Collection;
}
$parts = array_unique(explode(' ', $value));
$set = TagSplit::createSetFromSplits($this->user, $this->mapping, $parts);
$this->setCertainty(100);
return $set;
}
}

View File

@ -0,0 +1,235 @@
<?php
/**
* CsvProcessor.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\FileProcessor;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Object\ImportJournal;
use FireflyIII\Import\Specifics\SpecificInterface;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\TransactionJournalMeta;
use Illuminate\Support\Collection;
use Iterator;
use League\Csv\Reader;
use Log;
/**
* Class CsvProcessor, as the name suggests, goes over CSV file line by line and creates
* "ImportJournal" objects, which are used in another step to create new journals and transactions
* and what-not.
*
* @package FireflyIII\Import\FileProcessor
*/
class CsvProcessor implements FileProcessorInterface
{
/** @var ImportJob */
private $job;
/** @var Collection */
private $objects;
/** @var array */
private $validConverters = [];
/** @var array */
private $validSpecifics = [];
/**
* FileProcessorInterface constructor.
*/
public function __construct()
{
$this->objects = new Collection;
$this->validSpecifics = array_keys(config('csv.import_specifics'));
$this->validConverters = array_keys(config('csv.import_roles'));
}
/**
* @return Collection
*/
public function getObjects(): Collection
{
return $this->objects;
}
/**
* Does the actual job:
*
* @return bool
*/
public function run(): bool
{
Log::debug('Now in CsvProcessor run(). Job is now running...');
$entries = $this->getImportArray();
$index = 0;
Log::notice('Building importable objects from CSV file.');
foreach ($entries as $index => $row) {
// verify if not exists already:
if ($this->rowAlreadyImported($row)) {
$message = sprintf('Row #%d has already been imported.', $index);
$this->job->addError($index, $message);
$this->job->addStepsDone(5); // all steps.
Log::info($message);
continue;
}
$this->objects->push($this->importRow($index, $row));
$this->job->addStepsDone(1);
}
// if job has no step count, set it now:
$extended = $this->job->extended_status;
if ($extended['steps'] === 0) {
$extended['steps'] = $index * 5;
$this->job->extended_status = $extended;
$this->job->save();
}
return true;
}
/**
* @param ImportJob $job
*
* @return FileProcessorInterface
*/
public function setJob(ImportJob $job): FileProcessorInterface
{
$this->job = $job;
return $this;
}
/**
* Add meta data to the individual value and verify that it can be handled in a later stage.
*
* @param int $index
* @param string $value
*
* @return array
* @throws FireflyException
*/
private function annotateValue(int $index, string $value)
{
$value = trim($value);
$config = $this->job->configuration;
$role = $config['column-roles'][$index] ?? '_ignore';
$mapped = $config['column-mapping-config'][$index][$value] ?? null;
// throw error when not a valid converter.
if (!in_array($role, $this->validConverters)) {
throw new FireflyException(sprintf('"%s" is not a valid role.', $role));
}
$entry = [
'role' => $role,
'value' => $value,
'mapped' => $mapped,
];
return $entry;
}
/**
* @return Iterator
*/
private function getImportArray(): Iterator
{
$content = $this->job->uploadFileContents();
$config = $this->job->configuration;
$reader = Reader::createFromString($content);
$reader->setDelimiter($config['delimiter']);
$start = $config['has-headers'] ? 1 : 0;
$results = $reader->setOffset($start)->fetch();
Log::debug(sprintf('Created a CSV reader starting at offset %d', $start));
return $results;
}
/**
* Take a row, build import journal by annotating each value and storing it in the import journal.
*
* @param int $index
* @param array $row
*
* @return ImportJournal
*/
private function importRow(int $index, array $row): ImportJournal
{
Log::debug(sprintf('Now at row %d', $index));
$row = $this->specifics($row);
$journal = new ImportJournal;
$journal->setUser($this->job->user);
$journal->setHash(hash('sha256', json_encode($row)));
foreach ($row as $rowIndex => $value) {
$value = trim($value);
if (strlen($value) > 0) {
$annotated = $this->annotateValue($rowIndex, $value);
Log::debug('Annotated value', $annotated);
$journal->setValue($annotated);
}
}
Log::debug('ImportJournal complete, returning.');
return $journal;
}
/**
* Checks if the row has not been imported before.
*
* @param array $array
*
* @return bool
*/
private function rowAlreadyImported(array $array): bool
{
$string = json_encode($array);
$hash = hash('sha256', json_encode($string));
$json = json_encode($hash);
$entry = TransactionJournalMeta::leftJoin('transaction_journals', 'transaction_journals.id', '=', 'journal_meta.transaction_journal_id')
->where('data', $json)
->where('name', 'importHash')
->first();
if (!is_null($entry)) {
return true;
}
return false;
}
/**
* And this is the point where the specifix go to work.
*
* @param array $row
*
* @return array
* @throws FireflyException
*/
private function specifics(array $row): array
{
$config = $this->job->configuration;
//
foreach ($config['specifics'] as $name => $enabled) {
if (!in_array($name, $this->validSpecifics)) {
throw new FireflyException(sprintf('"%s" is not a valid class name', $name));
}
/** @var SpecificInterface $specific */
$specific = app('FireflyIII\Import\Specifics\\' . $name);
// it returns the row, possibly modified:
$row = $specific->run($row);
}
return $row;
}
}

View File

@ -0,0 +1,42 @@
<?php
/**
* FileProcessorInterface.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\FileProcessor;
use FireflyIII\Models\ImportJob;
use Illuminate\Support\Collection;
/**
* Interface FileProcessorInterface
*
* @package FireflyIII\Import\FileProcessor
*/
interface FileProcessorInterface
{
/**
* @return Collection
*/
public function getObjects(): Collection;
/**
* @return bool
*/
public function run(): bool;
/**
* @param ImportJob $job
*
* @return FileProcessorInterface
*/
public function setJob(ImportJob $job): FileProcessorInterface;
}

View File

@ -1,274 +0,0 @@
<?php
/**
* ImportEntry.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import;
use Carbon\Carbon;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* Class ImportEntry
*
* @package FireflyIII\Import
*/
class ImportEntry
{
/** @var array */
public $certain = [];
/** @var Collection */
public $errors;
/** @var string */
public $externalID;
/** @var array */
public $fields = [];
/** @var string */
public $hash;
/** @var User */
public $user;
/** @var bool */
public $valid = true;
/** @var int */
private $amountMultiplier = 0;
/** @var array */
private $validFields
= ['amount',
'date-transaction',
'date-interest',
'date-book',
'description',
'date-process',
'transaction-type',
'currency', 'asset-account', 'opposing-account', 'bill', 'budget', 'category', 'tags'];
/**
* ImportEntry constructor.
*/
public function __construct()
{
/** @var string $value */
foreach ($this->validFields as $value) {
$this->fields[$value] = null;
$this->certain[$value] = 0;
}
$this->errors = new Collection;
}
/**
* @param string $role
* @param int $certainty
* @param $convertedValue
*
* @throws FireflyException
*/
public function importValue(string $role, int $certainty, $convertedValue)
{
switch ($role) {
default:
Log::error('Import entry cannot handle object.', ['role' => $role]);
throw new FireflyException('Import entry cannot handle object of type "' . $role . '".');
case 'hash':
$this->hash = $convertedValue;
return;
case 'amount':
/*
* Easy enough.
*/
$this->setFloat('amount', $convertedValue, $certainty);
$this->applyMultiplier('amount'); // if present.
return;
case 'account-id':
case 'account-iban':
case 'account-name':
case 'account-number':
$this->setObject('asset-account', $convertedValue, $certainty);
break;
case 'opposing-number':
case 'opposing-iban':
case 'opposing-id':
case 'opposing-name':
$this->setObject('opposing-account', $convertedValue, $certainty);
break;
case 'bill-id':
case 'bill-name':
$this->setObject('bill', $convertedValue, $certainty);
break;
case 'budget-id':
case 'budget-name':
$this->setObject('budget', $convertedValue, $certainty);
break;
case 'category-id':
case 'category-name':
$this->setObject('category', $convertedValue, $certainty);
break;
case 'currency-code':
case 'currency-id':
case 'currency-name':
case 'currency-symbol':
$this->setObject('currency', $convertedValue, $certainty);
break;
case 'date-transaction':
$this->setDate('date-transaction', $convertedValue, $certainty);
break;
case 'date-interest':
$this->setDate('date-interest', $convertedValue, $certainty);
break;
case 'date-book':
$this->setDate('date-book', $convertedValue, $certainty);
break;
case 'date-process':
$this->setDate('date-process', $convertedValue, $certainty);
break;
case 'sepa-ct-id':
case 'sepa-db':
case 'sepa-ct-op':
case 'description':
$this->setAppendableString('description', $convertedValue);
break;
case '_ignore':
break;
case 'ing-debet-credit':
case 'rabo-debet-credit':
$this->manipulateFloat('amount', 'multiply', $convertedValue);
$this->applyMultiplier('amount'); // if present.
break;
case 'tags-comma':
case 'tags-space':
$this->appendCollection('tags', $convertedValue);
break;
case 'external-id':
$this->externalID = $convertedValue;
break;
}
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
}
/**
* @param string $field
* @param Collection $convertedValue
*/
private function appendCollection(string $field, Collection $convertedValue)
{
if (is_null($this->fields[$field])) {
$this->fields[$field] = new Collection;
}
$this->fields[$field] = $this->fields[$field]->merge($convertedValue);
}
/**
* @param string $field
*/
private function applyMultiplier(string $field)
{
if ($this->fields[$field] != 0 && $this->amountMultiplier != 0) {
$this->fields[$field] = $this->fields[$field] * $this->amountMultiplier;
}
}
/**
* @param string $field
* @param string $action
* @param $convertedValue
*
* @throws FireflyException
*/
private function manipulateFloat(string $field, string $action, $convertedValue)
{
switch ($action) {
default:
Log::error('Cannot handle manipulateFloat', ['field' => $field, 'action' => $action]);
throw new FireflyException('Cannot manipulateFloat with action ' . $action);
case 'multiply':
$this->amountMultiplier = $convertedValue;
break;
}
}
/**
* @param string $field
* @param string $value
*/
private function setAppendableString(string $field, string $value)
{
$value = trim($value);
$this->fields[$field] .= ' ' . $value;
}
/**
* @param string $field
* @param Carbon $date
* @param int $certainty
*/
private function setDate(string $field, Carbon $date, int $certainty)
{
if ($certainty > $this->certain[$field] && !is_null($date)) {
Log::debug(sprintf('ImportEntry: %s is now %s with certainty %d', $field, $date->format('Y-m-d'), $certainty));
$this->fields[$field] = $date;
$this->certain[$field] = $certainty;
return;
}
Log::info(sprintf('Will not set %s based on certainty %d (current certainty is %d) or NULL id.', $field, $certainty, $this->certain[$field]));
}
/**
* @param string $field
* @param float $value
* @param int $certainty
*/
private function setFloat(string $field, float $value, int $certainty)
{
if ($certainty > $this->certain[$field]) {
Log::debug(sprintf('ImportEntry: %s is now %f with certainty %d', $field, $value, $certainty));
$this->fields[$field] = $value;
$this->certain[$field] = $certainty;
return;
}
Log::info(sprintf('Will not set %s based on certainty %d (current certainty is %d).', $field, $certainty, $this->certain[$field]));
}
/**
* @param string $field
* @param $object
* @param int $certainty
*/
private function setObject(string $field, $object, int $certainty)
{
if ($certainty > $this->certain[$field] && !is_null($object->id)) {
Log::debug(sprintf('ImportEntry: %s ID is now %d with certainty %d', $field, $object->id, $certainty));
$this->fields[$field] = $object;
$this->certain[$field] = $certainty;
return;
}
Log::info(sprintf('Will not set %s based on certainty %d (current certainty is %d) or NULL id.', $field, $certainty, $this->certain[$field]));
}
}

View File

@ -1,85 +0,0 @@
<?php
/**
* ImportProcedure.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import;
use FireflyIII\Import\Importer\ImporterInterface;
use FireflyIII\Models\ImportJob;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use Illuminate\Support\Collection;
/**
* Class ImportProcedure
*
* @package FireflyIII\Import
*/
class ImportProcedure implements ImportProcedureInterface
{
/**
* @param ImportJob $job
*
* @return Collection
*/
public function runImport(ImportJob $job): Collection
{
// update job to say we started.
$job->status = 'import_running';
$job->save();
// create Importer
$valid = array_keys(config('firefly.import_formats'));
$class = 'INVALID';
if (in_array($job->file_type, $valid)) {
$class = config('firefly.import_formats.' . $job->file_type);
}
/** @var ImporterInterface $importer */
$importer = app($class);
$importer->setJob($job);
// create import entries
$collection = $importer->createImportEntries();
// validate / clean collection:
$validator = new ImportValidator($collection);
$validator->setUser($job->user);
$validator->setJob($job);
if ($job->configuration['import-account'] != 0) {
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($job->user);
$validator->setDefaultImportAccount($repository->find($job->configuration['import-account']));
}
$cleaned = $validator->clean();
// then import collection:
$storage = new ImportStorage($job->user, $cleaned);
$storage->setJob($job);
// and run store routine:
$result = $storage->store();
// grab import tag:
$status = $job->extended_status;
$status['importTag'] = $storage->importTag->id;
$job->extended_status = $status;
$job->status = 'import_complete';
$job->save();
return $result;
}
}

View File

@ -1,33 +0,0 @@
<?php
/**
* ImportProcedureInterface.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import;
use FireflyIII\Models\ImportJob;
use Illuminate\Support\Collection;
/**
* Interface ImportProcedureInterface
*
* @package FireflyIII\Import
*/
interface ImportProcedureInterface
{
/**
* @param ImportJob $job
*
* @return Collection
*/
public function runImport(ImportJob $job): Collection;
}

View File

@ -1,418 +0,0 @@
<?php
/**
* ImportStorage.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import;
use Carbon\Carbon;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\Rule;
use FireflyIII\Models\Tag;
use FireflyIII\Models\Transaction;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\Models\TransactionJournalMeta;
use FireflyIII\Models\TransactionType;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
use FireflyIII\Rules\Processor;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
use Steam;
/**
* Class ImportStorage
*
* @package FireflyIII\Import
*/
class ImportStorage
{
/** @var Collection */
public $entries;
/** @var Tag */
public $importTag;
/** @var ImportJob */
public $job;
/** @var User */
public $user;
/** @var Collection */
private $rules;
/**
* ImportStorage constructor.
*
* @param User $user
* @param Collection $entries
*/
public function __construct(User $user, Collection $entries)
{
$this->entries = $entries;
$this->user = $user;
$this->rules = $this->getUserRules();
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
}
/**
* @return Collection
*/
public function store(): Collection
{
// create a tag to join the transactions.
$this->importTag = $this->createImportTag();
$collection = new Collection;
Log::notice(sprintf('Started storing %d entry(ies).', $this->entries->count()));
foreach ($this->entries as $index => $entry) {
Log::debug(sprintf('--- import store start for row %d ---', $index));
// store entry:
$journal = $this->storeSingle($index, $entry);
$this->job->addStepsDone(1);
// apply rules:
$this->applyRules($journal);
$this->job->addStepsDone(1);
$collection->put($index, $journal);
}
Log::notice(sprintf('Finished storing %d entry(ies).', $collection->count()));
return $collection;
}
/**
* @param string $hash
*
* @return TransactionJournal
*/
private function alreadyImported(string $hash): TransactionJournal
{
$meta = TransactionJournalMeta
::leftJoin('transaction_journals', 'transaction_journals.id', '=', 'journal_meta.transaction_journal_id')
->where('journal_meta.name', 'originalImportHash')
->where('transaction_journals.user_id', $this->user->id)
->where('journal_meta.data', json_encode($hash))->first(['journal_meta.*']);
if (!is_null($meta)) {
/** @var TransactionJournal $journal */
$journal = $meta->transactionjournal;
if (intval($journal->completed) === 1) {
return $journal;
}
}
return new TransactionJournal;
}
/**
* @param TransactionJournal $journal
*
* @return bool
*/
private function applyRules(TransactionJournal $journal): bool
{
if ($this->rules->count() > 0) {
/** @var Rule $rule */
foreach ($this->rules as $rule) {
Log::debug(sprintf('Going to apply rule #%d to journal %d.', $rule->id, $journal->id));
$processor = Processor::make($rule);
$processor->handleTransactionJournal($journal);
if ($rule->stop_processing) {
return true;
}
}
}
return true;
}
/**
* @return Tag
*/
private function createImportTag(): Tag
{
/** @var TagRepositoryInterface $repository */
$repository = app(TagRepositoryInterface::class);
$repository->setUser($this->user);
$data = [
'tag' => trans('firefly.import_with_key', ['key' => $this->job->key]),
'date' => new Carbon,
'description' => null,
'latitude' => null,
'longitude' => null,
'zoomLevel' => null,
'tagMode' => 'nothing',
];
$tag = $repository->store($data);
return $tag;
}
/**
* @return Collection
*/
private function getUserRules(): Collection
{
$set = Rule::distinct()
->where('rules.user_id', $this->user->id)
->leftJoin('rule_groups', 'rule_groups.id', '=', 'rules.rule_group_id')
->leftJoin('rule_triggers', 'rules.id', '=', 'rule_triggers.rule_id')
->where('rule_groups.active', 1)
->where('rule_triggers.trigger_type', 'user_action')
->where('rule_triggers.trigger_value', 'store-journal')
->where('rules.active', 1)
->orderBy('rule_groups.order', 'ASC')
->orderBy('rules.order', 'ASC')
->get(['rules.*', 'rule_groups.order']);
Log::debug(sprintf('Found %d user rules.', $set->count()));
return $set;
}
/**
* @param $entry
*
* @return array
* @throws FireflyException
*/
private function storeAccounts($entry): array
{
// then create transactions. Single ones, unfortunately.
switch ($entry->fields['transaction-type']->type) {
default:
throw new FireflyException('ImportStorage cannot handle ' . $entry->fields['transaction-type']->type);
case TransactionType::WITHDRAWAL:
$source = $entry->fields['asset-account'];
$destination = $entry->fields['opposing-account'];
// make amount positive, if it is not.
break;
case TransactionType::DEPOSIT:
$source = $entry->fields['opposing-account'];
$destination = $entry->fields['asset-account'];
break;
case TransactionType::TRANSFER:
// depends on amount:
if ($entry->fields['amount'] < 0) {
$source = $entry->fields['asset-account'];
$destination = $entry->fields['opposing-account'];
break;
}
$destination = $entry->fields['asset-account'];
$source = $entry->fields['opposing-account'];
break;
}
return [
'source' => $source,
'destination' => $destination,
];
}
/**
* @param TransactionJournal $journal
* @param ImportEntry $entry
*/
private function storeBill(TransactionJournal $journal, ImportEntry $entry)
{
if (!is_null($entry->fields['bill']) && !is_null($entry->fields['bill']->id)) {
$journal->bill()->associate($entry->fields['bill']);
Log::debug('Attached bill', ['id' => $entry->fields['bill']->id, 'name' => $entry->fields['bill']->name]);
$journal->save();
}
}
/**
* @param TransactionJournal $journal
* @param ImportEntry $entry
*/
private function storeBudget(TransactionJournal $journal, ImportEntry $entry)
{
if (!is_null($entry->fields['budget']) && !is_null($entry->fields['budget']->id)) {
$journal->budgets()->save($entry->fields['budget']);
Log::debug('Attached budget', ['id' => $entry->fields['budget']->id, 'name' => $entry->fields['budget']->name]);
$journal->save();
}
}
/**
* @param TransactionJournal $journal
* @param ImportEntry $entry
*/
private function storeCategory(TransactionJournal $journal, ImportEntry $entry)
{
if (!is_null($entry->fields['category']) && !is_null($entry->fields['category']->id)) {
$journal->categories()->save($entry->fields['category']);
Log::debug('Attached category', ['id' => $entry->fields['category']->id, 'name' => $entry->fields['category']->name]);
$journal->save();
}
}
/**
* @param $entry
*
* @return TransactionJournal
*/
private function storeJournal($entry): TransactionJournal
{
$billId = is_null($entry->fields['bill']) || intval($entry->fields['bill']->id) === 0 ? null : intval($entry->fields['bill']->id);
$journalData = [
'user_id' => $entry->user->id,
'transaction_type_id' => $entry->fields['transaction-type']->id,
'bill_id' => $billId,
'transaction_currency_id' => $entry->fields['currency']->id,
'description' => $entry->fields['description'],
'date' => $entry->fields['date-transaction'],
'interest_date' => $entry->fields['date-interest'],
'book_date' => $entry->fields['date-book'],
'process_date' => $entry->fields['date-process'],
'completed' => 0,
];
/** @var TransactionJournal $journal */
$journal = TransactionJournal::create($journalData);
foreach ($journal->getErrors()->all() as $err) {
Log::error('Error when storing journal: ' . $err);
}
Log::debug('Created journal', ['id' => $journal->id]);
// save hash as meta value:
$meta = new TransactionJournalMeta;
$meta->name = 'originalImportHash';
$meta->data = $entry->hash;
$meta->transactionJournal()->associate($journal);
$meta->save();
return $journal;
}
/**
* @param int $index
* @param ImportEntry $entry
*
* @return TransactionJournal
* @throws FireflyException
*/
private function storeSingle(int $index, ImportEntry $entry): TransactionJournal
{
if ($entry->valid === false) {
Log::warning(sprintf('Cannot import row %d, because the entry is not valid.', $index));
$errors = join(', ', $entry->errors->all());
$errorText = sprintf('Row #%d: ' . $errors, $index);
$extendedStatus = $this->job->extended_status;
$extendedStatus['errors'][] = $errorText;
$this->job->extended_status = $extendedStatus;
$this->job->save();
return new TransactionJournal;
}
$alreadyImported = $this->alreadyImported($entry->hash);
if (!is_null($alreadyImported->id)) {
Log::warning(sprintf('Cannot import row %d, because it has already been imported (journal #%d).', $index, $alreadyImported->id));
$errorText = trans(
'firefly.import_double',
['row' => $index, 'link' => route('transactions.show', [$alreadyImported->id]), 'description' => $alreadyImported->description]
);
$extendedStatus = $this->job->extended_status;
$extendedStatus['errors'][] = $errorText;
$this->job->extended_status = $extendedStatus;
$this->job->save();
return new TransactionJournal;
}
Log::debug(sprintf('Going to store row %d', $index));
$journal = $this->storeJournal($entry);
$amount = Steam::positive($entry->fields['amount']);
$accounts = $this->storeAccounts($entry);
// create new transactions. This is something that needs a rewrite for multiple/split transactions.
$sourceData = [
'account_id' => $accounts['source']->id,
'transaction_journal_id' => $journal->id,
'description' => null,
'amount' => bcmul($amount, '-1'),
];
$destinationData = [
'account_id' => $accounts['destination']->id,
'transaction_journal_id' => $journal->id,
'description' => null,
'amount' => $amount,
];
$one = Transaction::create($sourceData);
$two = Transaction::create($destinationData);
$error = false;
if (is_null($one->id)) {
Log::error('Could not create transaction 1.', $one->getErrors()->all());
$error = true;
}
if (is_null($two->id)) {
Log::error('Could not create transaction 1.', $two->getErrors()->all());
$error = true;
}
// respond to error
if ($error === true) {
$errorText = sprintf('Cannot import row %d, because an error occured when storing data.', $index);
Log::error($errorText);
$extendedStatus = $this->job->extended_status;
$extendedStatus['errors'][] = $errorText;
$this->job->extended_status = $extendedStatus;
$this->job->save();
return new TransactionJournal;
}
Log::debug('Created transaction 1', ['id' => $one->id, 'account' => $one->account_id, 'account_name' => $accounts['source']->name]);
Log::debug('Created transaction 2', ['id' => $two->id, 'account' => $two->account_id, 'account_name' => $accounts['destination']->name]);
$journal->completed = 1;
$journal->save();
// attach import tag.
$journal->tags()->save($this->importTag);
// now attach budget and so on.
$this->storeBudget($journal, $entry);
$this->storeCategory($journal, $entry);
$this->storeBill($journal, $entry);
return $journal;
}
}

View File

@ -1,438 +0,0 @@
<?php
/**
* ImportValidator.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import;
use Carbon\Carbon;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\TransactionType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
use Preferences;
/**
* Class ImportValidator
*
* @package FireflyIII\Import
*/
class ImportValidator
{
/** @var ImportJob */
public $job;
/** @var Account */
protected $defaultImportAccount;
/** @var Collection */
protected $entries;
/** @var User */
protected $user;
/**
* ImportValidator constructor.
*
* @param Collection $entries
*/
public function __construct(Collection $entries)
{
$this->entries = $entries;
}
/**
* Clean collection by filling in all the blanks.
*/
public function clean(): Collection
{
Log::notice(sprintf('Started validating %d entry(ies).', $this->entries->count()));
$newCollection = new Collection;
/** @var ImportEntry $entry */
foreach ($this->entries as $index => $entry) {
Log::debug(sprintf('--- import validator start for row %d ---', $index));
/*
* X Adds the date (today) if no date is present.
* X Determins the types of accounts involved (asset, expense, revenue).
* X Determins the type of transaction (withdrawal, deposit, transfer).
* - Determins the currency of the transaction.
* X Adds a default description if there isn't one present.
*/
$entry = $this->checkAmount($entry);
$entry = $this->setDate($entry);
$entry = $this->setAssetAccount($entry);
$entry = $this->setOpposingAccount($entry);
$entry = $this->cleanDescription($entry);
$entry = $this->setTransactionType($entry);
$entry = $this->setTransactionCurrency($entry);
$newCollection->put($index, $entry);
$this->job->addStepsDone(1);
}
Log::notice(sprintf('Finished validating %d entry(ies).', $newCollection->count()));
return $newCollection;
}
/**
* @param Account $defaultImportAccount
*/
public function setDefaultImportAccount(Account $defaultImportAccount)
{
$this->defaultImportAccount = $defaultImportAccount;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function checkAmount(ImportEntry $entry): ImportEntry
{
if ($entry->fields['amount'] == 0) {
$entry->valid = false;
$entry->errors->push('Amount of transaction is zero, cannot handle.');
Log::warning('Amount of transaction is zero, cannot handle.');
return $entry;
}
Log::debug('Amount is OK.');
return $entry;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function cleanDescription(ImportEntry $entry): ImportEntry
{
if (!isset($entry->fields['description'])) {
Log::debug('Set empty transaction description because field was not set.');
$entry->fields['description'] = '(empty transaction description)';
return $entry;
}
if (is_null($entry->fields['description'])) {
Log::debug('Set empty transaction description because field was null.');
$entry->fields['description'] = '(empty transaction description)';
return $entry;
}
$entry->fields['description'] = trim($entry->fields['description']);
if (strlen($entry->fields['description']) == 0) {
Log::debug('Set empty transaction description because field was empty.');
$entry->fields['description'] = '(empty transaction description)';
return $entry;
}
Log::debug('Transaction description is OK.', ['description' => $entry->fields['description']]);
return $entry;
}
/**
* @param Account $account
* @param string $type
*
* @return Account
*/
private function convertAccount(Account $account, string $type): Account
{
$accountType = $account->accountType->type;
if ($accountType === $type) {
Log::debug(sprintf('Account %s already of type %s', $account->name, $type));
return $account;
}
// find it first by new type:
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
$result = $repository->findByName($account->name, [$type]);
if (is_null($result->id)) {
// can convert account:
Log::debug(sprintf('No account named %s of type %s, create new account.', $account->name, $type));
$result = $repository->store(
[
'user' => $this->user->id,
'accountType' => config('firefly.shortNamesByFullName.' . $type),
'name' => $account->name,
'virtualBalance' => 0,
'active' => true,
'iban' => null,
'openingBalance' => 0,
]
);
}
Log::debug(
sprintf(
'Using another account named %s (#%d) of type %s, will use that one instead of %s (#%d)', $account->name, $result->id, $type, $account->name,
$account->id
)
);
return $result;
}
/**
* @return Account
*/
private function fallbackExpenseAccount(): Account
{
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
$name = 'Unknown expense account';
$result = $repository->findByName($name, [AccountType::EXPENSE]);
if (is_null($result->id)) {
$result = $repository->store(
['name' => $name, 'iban' => null, 'openingBalance' => 0, 'user' => $this->user->id, 'accountType' => 'expense', 'virtualBalance' => 0,
'active' => true]
);
}
return $result;
}
/**
* @return Account
*/
private function fallbackRevenueAccount(): Account
{
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$repository->setUser($this->user);
$name = 'Unknown revenue account';
$result = $repository->findByName($name, [AccountType::REVENUE]);
if (is_null($result->id)) {
$result = $repository->store(
['name' => $name, 'iban' => null, 'openingBalance' => 0, 'user' => $this->user->id, 'accountType' => 'revenue', 'virtualBalance' => 0,
'active' => true]
);
}
return $result;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function setAssetAccount(ImportEntry $entry): ImportEntry
{
if (is_null($entry->fields['asset-account'])) {
if (!is_null($this->defaultImportAccount)) {
Log::debug('Set asset account from default asset account');
$entry->fields['asset-account'] = $this->defaultImportAccount;
return $entry;
}
// default import is null? should not happen. Entry cannot be imported.
// set error message and block.
$entry->valid = false;
Log::warning('Cannot import entry. Asset account is NULL and import account is also NULL.');
return $entry;
}
Log::debug('Asset account is OK.', ['id' => $entry->fields['asset-account']->id, 'name' => $entry->fields['asset-account']->name]);
return $entry;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function setDate(ImportEntry $entry): ImportEntry
{
if (is_null($entry->fields['date-transaction']) || $entry->certain['date-transaction'] == 0) {
// empty date field? find alternative.
$alternatives = ['date-book', 'date-interest', 'date-process'];
foreach ($alternatives as $alternative) {
if (!is_null($entry->fields[$alternative])) {
Log::debug(sprintf('Copied date-transaction from %s.', $alternative));
$entry->fields['date-transaction'] = clone $entry->fields[$alternative];
return $entry;
}
}
// date is still null at this point
Log::debug('Set date-transaction to today.');
$entry->fields['date-transaction'] = new Carbon;
return $entry;
}
// confidence is zero?
Log::debug('Date-transaction is OK');
return $entry;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function setOpposingAccount(ImportEntry $entry): ImportEntry
{
// empty opposing account. Create one based on amount.
if (is_null($entry->fields['opposing-account'])) {
if ($entry->fields['amount'] < 0) {
// create or find general opposing expense account.
Log::debug('Created fallback expense account');
$entry->fields['opposing-account'] = $this->fallbackExpenseAccount();
return $entry;
}
Log::debug('Created fallback revenue account');
$entry->fields['opposing-account'] = $this->fallbackRevenueAccount();
return $entry;
}
// opposing is of type "import". Convert to correct type (by amount):
$type = $entry->fields['opposing-account']->accountType->type;
if ($type == AccountType::IMPORT && $entry->fields['amount'] < 0) {
$account = $this->convertAccount($entry->fields['opposing-account'], AccountType::EXPENSE);
$entry->fields['opposing-account'] = $account;
Log::debug('Converted import account to expense account');
return $entry;
}
if ($type == AccountType::IMPORT && $entry->fields['amount'] > 0) {
$account = $this->convertAccount($entry->fields['opposing-account'], AccountType::REVENUE);
$entry->fields['opposing-account'] = $account;
Log::debug('Converted import account to revenue account');
return $entry;
}
// amount < 0, but opposing is revenue
if ($type == AccountType::REVENUE && $entry->fields['amount'] < 0) {
$account = $this->convertAccount($entry->fields['opposing-account'], AccountType::EXPENSE);
$entry->fields['opposing-account'] = $account;
Log::debug('Converted revenue account to expense account');
return $entry;
}
// amount > 0, but opposing is expense
if ($type == AccountType::EXPENSE && $entry->fields['amount'] > 0) {
$account = $this->convertAccount($entry->fields['opposing-account'], AccountType::REVENUE);
$entry->fields['opposing-account'] = $account;
Log::debug('Converted expense account to revenue account');
return $entry;
}
// account type is OK
Log::debug('Opposing account is OK.');
return $entry;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function setTransactionCurrency(ImportEntry $entry): ImportEntry
{
if (is_null($entry->fields['currency'])) {
/** @var CurrencyRepositoryInterface $repository */
$repository = app(CurrencyRepositoryInterface::class);
$repository->setUser($this->user);
// is the default currency for the user or the system
$defaultCode = Preferences::getForUser($this->user, 'currencyPreference', config('firefly.default_currency', 'EUR'))->data;
$entry->fields['currency'] = $repository->findByCode($defaultCode);
Log::debug(sprintf('Set currency to %s', $defaultCode));
return $entry;
}
Log::debug(sprintf('Currency is OK: %s', $entry->fields['currency']->code));
return $entry;
}
/**
* @param ImportEntry $entry
*
* @return ImportEntry
*/
private function setTransactionType(ImportEntry $entry): ImportEntry
{
Log::debug(sprintf('Opposing account is of type %s', $entry->fields['opposing-account']->accountType->type));
$type = $entry->fields['opposing-account']->accountType->type;
switch ($type) {
case AccountType::EXPENSE:
$entry->fields['transaction-type'] = TransactionType::whereType(TransactionType::WITHDRAWAL)->first();
Log::debug('Transaction type is now withdrawal.');
return $entry;
case AccountType::REVENUE:
$entry->fields['transaction-type'] = TransactionType::whereType(TransactionType::DEPOSIT)->first();
Log::debug('Transaction type is now deposit.');
return $entry;
case AccountType::DEFAULT:
case AccountType::ASSET:
$entry->fields['transaction-type'] = TransactionType::whereType(TransactionType::TRANSFER)->first();
Log::debug('Transaction type is now transfer.');
return $entry;
}
Log::warning(sprintf('Opposing account is of type %s, cannot handle this.', $type));
$entry->valid = false;
$entry->errors->push(sprintf('Opposing account is of type %s, cannot handle this.', $type));
return $entry;
}
}

View File

@ -1,175 +0,0 @@
<?php
/**
* CsvImporter.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Importer;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Converter\ConverterInterface;
use FireflyIII\Import\ImportEntry;
use FireflyIII\Import\Specifics\SpecificInterface;
use FireflyIII\Models\ImportJob;
use Illuminate\Support\Collection;
use League\Csv\Reader;
use Log;
/**
* Class CsvImporter
*
* @package FireflyIII\Import\Importer
*/
class CsvImporter implements ImporterInterface
{
/** @var Collection */
public $collection;
/** @var ImportJob */
public $job;
public $validSpecifics = [];
/**
* CsvImporter constructor.
*/
public function __construct()
{
$this->collection = new Collection;
$this->validSpecifics = array_keys(config('csv.import_specifics'));
}
/**
* Run the actual import
*
* @return Collection
*/
public function createImportEntries(): Collection
{
$config = $this->job->configuration;
$content = $this->job->uploadFileContents();
// create CSV reader.
$reader = Reader::createFromString($content);
$reader->setDelimiter($config['delimiter']);
$start = $config['has-headers'] ? 1 : 0;
$results = $reader->fetch();
Log::notice('Building importable objects from CSV file.');
foreach ($results as $index => $row) {
if ($index >= $start) {
$line = $index + 1;
Log::debug('----- import entry build start --');
Log::debug(sprintf('Now going to import row %d.', $index));
$importEntry = $this->importSingleRow($index, $row);
$this->collection->put($line, $importEntry);
/**
* 1. Build import entry.
* 2. Validate import entry.
* 3. Store journal.
* 4. Run rules.
*/
$this->job->addTotalSteps(4);
$this->job->addStepsDone(1);
}
}
Log::debug(sprintf('Import collection contains %d entries', $this->collection->count()));
Log::notice(sprintf('Built %d importable object(s) from your CSV file.', $this->collection->count()));
return $this->collection;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
}
/**
* @param int $index
* @param array $row
*
* @return ImportEntry
* @throws FireflyException
*/
private function importSingleRow(int $index, array $row): ImportEntry
{
// create import object. This is where each entry ends up.
$object = new ImportEntry;
Log::debug(sprintf('Now at row %d', $index));
// set some vars:
$object->setUser($this->job->user);
$config = $this->job->configuration;
$json = json_encode($row);
if ($json === false) {
throw new FireflyException(sprintf('Could not process row #%d. Are you sure the uploaded file is encoded as "UTF-8"?', $index));
}
// hash the row:
$hash = hash('sha256', $json);
$object->importValue('hash', 100, $hash);
// and this is the point where the specifix go to work.
foreach ($config['specifics'] as $name => $enabled) {
if (!in_array($name, $this->validSpecifics)) {
throw new FireflyException(sprintf('"%s" is not a valid class name', $name));
}
/** @var SpecificInterface $specific */
$specific = app('FireflyIII\Import\Specifics\\' . $name);
// it returns the row, possibly modified:
$row = $specific->run($row);
}
foreach ($row as $rowIndex => $value) {
// find the role for this column:
$role = $config['column-roles'][$rowIndex] ?? '_ignore';
$doMap = $config['column-do-mapping'][$rowIndex] ?? false;
$validConverters = array_keys(config('csv.import_roles'));
// throw error when not a valid converter.
if (!in_array($role, $validConverters)) {
throw new FireflyException(sprintf('"%s" is not a valid role.', $role));
}
$converterClass = config('csv.import_roles.' . $role . '.converter');
$mapping = $config['column-mapping-config'][$rowIndex] ?? [];
$className = sprintf('FireflyIII\\Import\\Converter\\%s', $converterClass);
/** @var ConverterInterface $converter */
$converter = app($className);
// set some useful values for the converter:
$converter->setMapping($mapping);
$converter->setDoMap($doMap);
$converter->setUser($this->job->user);
$converter->setConfig($config);
// run the converter for this value:
$convertedValue = $converter->convert($value);
$certainty = $converter->getCertainty();
// log it.
Log::debug('Value ', ['index' => $rowIndex, 'value' => $value, 'role' => $role]);
// store in import entry:
Log::debug('Going to import', ['role' => $role, 'value' => $value, 'certainty' => $certainty]);
$object->importValue($role, $certainty, $convertedValue);
}
return $object;
}
}

View File

@ -1,38 +0,0 @@
<?php
/**
* ImporterInterface.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Importer;
use FireflyIII\Models\ImportJob;
use Illuminate\Support\Collection;
/**
* Interface ImporterInterface
*
* @package FireflyIII\Import\Importer
*/
interface ImporterInterface
{
/**
* Run the actual import
*
* @return Collection
*/
public function createImportEntries(): Collection;
/**
* @param ImportJob $job
*
*/
public function setJob(ImportJob $job);
}

View File

@ -50,7 +50,7 @@ class AssetAccountIbans implements MapperInterface
asort($list);
$list = $topList + $list;
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -47,7 +47,7 @@ class AssetAccounts implements MapperInterface
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -40,7 +40,7 @@ class Bills implements MapperInterface
}
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -41,7 +41,7 @@ class Budgets implements MapperInterface
}
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -41,7 +41,7 @@ class Categories implements MapperInterface
}
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -56,7 +56,7 @@ class OpposingAccountIbans implements MapperInterface
asort($list);
$list = $topList + $list;
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -53,7 +53,7 @@ class OpposingAccounts implements MapperInterface
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;
}

View File

@ -40,7 +40,7 @@ class Tags implements MapperInterface
}
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -36,7 +36,7 @@ class TransactionCurrencies implements MapperInterface
asort($list);
$list = [0 => trans('csv.do_not_map')] + $list;
$list = [0 => trans('csv.map_do_not_map')] + $list;
return $list;

View File

@ -0,0 +1,299 @@
<?php
/**
* ImportAccount.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* Class ImportAccount
*
* @package FireflyIII\Import\Object
*/
class ImportAccount
{
/** @var Account */
private $account;
/** @var array */
private $accountIban = [];
/** @var array */
private $accountId = [];
/** @var array */
private $accountName = [];
/** @var array */
private $accountNumber = [];
/** @var string */
private $expectedType = '';
/** @var AccountRepositoryInterface */
private $repository;
/** @var User */
private $user;
/**
* ImportAccount constructor.
*/
public function __construct()
{
$this->expectedType = AccountType::ASSET;
$this->account = new Account;
$this->repository = app(AccountRepositoryInterface::class);
Log::debug('Created ImportAccount.');
}
/**
* @return Account
*/
public function getAccount(): Account
{
if (is_null($this->account->id)) {
$this->store();
}
return $this->account;
}
/**
* @param array $accountIban
*/
public function setAccountIban(array $accountIban)
{
$this->accountIban = $accountIban;
}
/**
* @param array $value
*/
public function setAccountId(array $value)
{
$this->accountId = $value;
}
/**
* @param array $accountName
*/
public function setAccountName(array $accountName)
{
$this->accountName = $accountName;
}
/**
* @param array $accountNumber
*/
public function setAccountNumber(array $accountNumber)
{
$this->accountNumber = $accountNumber;
}
/**
* @param string $expectedType
*/
public function setExpectedType(string $expectedType)
{
$this->expectedType = $expectedType;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
$this->repository->setUser($user);
}
/**
* @return Account
*/
private function findExistingObject(): Account
{
Log::debug('In findExistingObject() for Account');
// 0: determin account type:
/** @var AccountType $accountType */
$accountType = AccountType::whereType($this->expectedType)->first();
// 1: find by ID, iban or name (and type)
if (count($this->accountId) === 3) {
Log::debug(sprintf('Finding account of type %d and ID %d', $accountType->id, $this->accountId['value']));
/** @var Account $account */
$account = $this->user->accounts()->where('account_type_id', $accountType->id)->where('id', $this->accountId['value'])->first();
if (!is_null($account)) {
Log::debug(sprintf('Found unmapped %s account by ID (#%d): %s', $this->expectedType, $account->id, $account->name));
return $account;
}
Log::debug('Found nothing.');
}
/** @var Collection $accounts */
$accounts = $this->repository->getAccountsByType([$accountType->type]);
// Two: find by IBAN (and type):
if (count($this->accountIban) === 3) {
$iban = $this->accountIban['value'];
Log::debug(sprintf('Finding account of type %d and IBAN %s', $accountType->id, $iban));
$filtered = $accounts->filter(
function (Account $account) use ($iban) {
if ($account->iban === $iban) {
Log::debug(
sprintf('Found unmapped %s account by IBAN (#%d): %s (%s)', $this->expectedType, $account->id, $account->name, $account->iban)
);
return $account;
}
return null;
}
);
if ($filtered->count() === 1) {
return $filtered->first();
}
Log::debug('Found nothing.');
}
// Three: find by name (and type):
if (count($this->accountName) === 3) {
$name = $this->accountName['value'];
Log::debug(sprintf('Finding account of type %d and name %s', $accountType->id, $name));
$filtered = $accounts->filter(
function (Account $account) use ($name) {
if ($account->name === $name) {
Log::debug(sprintf('Found unmapped %s account by name (#%d): %s', $this->expectedType, $account->id, $account->name));
return $account;
}
return null;
}
);
if ($filtered->count() === 1) {
return $filtered->first();
}
Log::debug('Found nothing.');
}
// 4: do not search by account number.
Log::debug('Found NO existing accounts.');
return new Account;
}
/**
* @return Account
*/
private function findMappedObject(): Account
{
Log::debug('In findMappedObject() for Account');
$fields = ['accountId', 'accountIban', 'accountNumber', 'accountName'];
foreach ($fields as $field) {
$array = $this->$field;
Log::debug(sprintf('Find mapped account based on field "%s" with value', $field), $array);
// check if a pre-mapped object exists.
$mapped = $this->getMappedObject($array);
if (!is_null($mapped->id)) {
Log::debug(sprintf('Found account #%d!', $mapped->id));
return $mapped;
}
}
Log::debug('Found no account on mapped data or no map present.');
return new Account;
}
/**
* @param array $array
*
* @return Account
*/
private function getMappedObject(array $array): Account
{
Log::debug('In getMappedObject() for Account');
if (count($array) === 0) {
Log::debug('Array is empty, nothing will come of this.');
return new Account;
}
if (array_key_exists('mapped', $array) && is_null($array['mapped'])) {
Log::debug(sprintf('No map present for value "%s". Return NULL.', $array['value']));
return new Account;
}
Log::debug('Finding a mapped account based on', $array);
$search = intval($array['mapped']);
$account = $this->repository->find($search);
Log::debug(sprintf('Found account! #%d ("%s"). Return it', $account->id, $account->name));
return $account;
}
/**
* @return bool
*/
private function store(): bool
{
// 1: find mapped object:
$mapped = $this->findMappedObject();
if (!is_null($mapped->id)) {
$this->account = $mapped;
return true;
}
// 2: find existing by given values:
$found = $this->findExistingObject();
if (!is_null($found->id)) {
$this->account = $found;
return true;
}
// 3: if found nothing, retry the search with an asset account:
Log::debug('Will try to find an asset account just in case.');
$oldExpectedType = $this->expectedType;
$this->expectedType = AccountType::ASSET;
$found = $this->findExistingObject();
if (!is_null($found->id)) {
Log::debug('Found asset account!');
$this->account = $found;
return true;
}
$this->expectedType = $oldExpectedType;
Log::debug(sprintf('Found no account of type %s so must create one ourselves.', $this->expectedType));
$data = [
'accountType' => config('firefly.shortNamesByFullName.' . $this->expectedType),
'name' => $this->accountName['value'] ?? '(no name)',
'iban' => $this->accountIban['value'] ?? null,
'active' => true,
'virtualBalance' => null,
];
$this->account = $this->repository->store($data);
Log::debug(sprintf('Successfully stored new account #%d: %s', $this->account->id, $this->account->name));
return true;
}
}

View File

@ -0,0 +1,228 @@
<?php
/**
* ImportBill.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Models\Bill;
use FireflyIII\Repositories\Bill\BillRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* Class ImportBill
*
* @package FireflyIII\Import\Object
*/
class ImportBill
{
/** @var Bill */
private $bill;
/** @var array */
private $id = [];
/** @var array */
private $name = [];
/** @var BillRepositoryInterface */
private $repository;
/** @var User */
private $user;
/**
* ImportBill constructor.
*/
public function __construct()
{
$this->bill = new Bill;
$this->repository = app(BillRepositoryInterface::class);
Log::debug('Created ImportBill.');
}
/**
* @return Bill
*/
public function getBill(): Bill
{
if (is_null($this->bill->id)) {
$this->store();
}
return $this->bill;
}
/**
* @param array $id
*/
public function setId(array $id)
{
$this->id = $id;
}
/**
* @param array $name
*/
public function setName(array $name)
{
$this->name = $name;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
$this->repository->setUser($user);
}
/**
* @return Bill
*/
private function findExistingObject(): Bill
{
Log::debug('In findExistingObject() for Bill');
// 1: find by ID, or name
if (count($this->id) === 3) {
Log::debug(sprintf('Finding bill with ID #%d', $this->id['value']));
/** @var Bill $bill */
$bill = $this->repository->find(intval($this->id['value']));
if (!is_null($bill->id)) {
Log::debug(sprintf('Found unmapped bill by ID (#%d): %s', $bill->id, $bill->name));
return $bill;
}
Log::debug('Found nothing.');
}
// 2: find by name
if (count($this->name) === 3) {
/** @var Collection $bills */
$bills = $this->repository->getBills();
$name = $this->name['value'];
Log::debug(sprintf('Finding bill with name %s', $name));
$filtered = $bills->filter(
function (Bill $bill) use ($name) {
if ($bill->name === $name) {
Log::debug(sprintf('Found unmapped bill by name (#%d): %s', $bill->id, $bill->name));
return $bill;
}
return null;
}
);
if ($filtered->count() === 1) {
return $filtered->first();
}
Log::debug('Found nothing.');
}
// 4: do not search by account number.
Log::debug('Found NO existing bills.');
return new Bill;
}
/**
* @return Bill
*/
private function findMappedObject(): Bill
{
Log::debug('In findMappedObject() for Bill');
$fields = ['id', 'name'];
foreach ($fields as $field) {
$array = $this->$field;
Log::debug(sprintf('Find mapped bill based on field "%s" with value', $field), $array);
// check if a pre-mapped object exists.
$mapped = $this->getMappedObject($array);
if (!is_null($mapped->id)) {
Log::debug(sprintf('Found bill #%d!', $mapped->id));
return $mapped;
}
}
Log::debug('Found no bill on mapped data or no map present.');
return new Bill;
}
/**
* @param array $array
*
* @return Bill
*/
private function getMappedObject(array $array): Bill
{
Log::debug('In getMappedObject() for Bill');
if (count($array) === 0) {
Log::debug('Array is empty, nothing will come of this.');
return new Bill;
}
if (array_key_exists('mapped', $array) && is_null($array['mapped'])) {
Log::debug(sprintf('No map present for value "%s". Return NULL.', $array['value']));
return new Bill;
}
Log::debug('Finding a mapped bill based on', $array);
$search = intval($array['mapped']);
$account = $this->repository->find($search);
Log::debug(sprintf('Found bill! #%d ("%s"). Return it', $account->id, $account->name));
return $account;
}
/**
* @return bool
*/
private function store(): bool
{
// 1: find mapped object:
$mapped = $this->findMappedObject();
if (!is_null($mapped->id)) {
$this->bill = $mapped;
return true;
}
// 2: find existing by given values:
$found = $this->findExistingObject();
if (!is_null($found->id)) {
$this->bill = $found;
return true;
}
$name = $this->name['value'] ?? '';
if (strlen($name) === 0) {
return true;
}
Log::debug('Found no bill so must create one ourselves.');
$data = [
'name' => $name,
];
$this->bill = $this->repository->store($data);
Log::debug(sprintf('Successfully stored new bill #%d: %s', $this->bill->id, $this->bill->name));
return true;
}
}

View File

@ -0,0 +1,229 @@
<?php
/**
* ImportBudget.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Models\Budget;
use FireflyIII\Repositories\Budget\BudgetRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* Class ImportBudget
*
* @package FireflyIII\Import\Object
*/
class ImportBudget
{
/** @var Budget */
private $budget;
/** @var array */
private $id = [];
/** @var array */
private $name = [];
/** @var BudgetRepositoryInterface */
private $repository;
/** @var User */
private $user;
/**
* ImportBudget constructor.
*/
public function __construct()
{
$this->budget = new Budget;
$this->repository = app(BudgetRepositoryInterface::class);
Log::debug('Created ImportBudget.');
}
/**
* @return Budget
*/
public function getBudget(): Budget
{
if (is_null($this->budget->id)) {
$this->store();
}
return $this->budget;
}
/**
* @param array $id
*/
public function setId(array $id)
{
$this->id = $id;
}
/**
* @param array $name
*/
public function setName(array $name)
{
$this->name = $name;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
$this->repository->setUser($user);
}
/**
* @return Budget
*/
private function findExistingObject(): Budget
{
Log::debug('In findExistingObject() for Budget');
// 1: find by ID, or name
if (count($this->id) === 3) {
Log::debug(sprintf('Finding budget with ID #%d', $this->id['value']));
/** @var Budget $budget */
$budget = $this->repository->find(intval($this->id['value']));
if (!is_null($budget->id)) {
Log::debug(sprintf('Found unmapped budget by ID (#%d): %s', $budget->id, $budget->name));
return $budget;
}
Log::debug('Found nothing.');
}
// 2: find by name
if (count($this->name) === 3) {
/** @var Collection $budgets */
$budgets = $this->repository->getBudgets();
$name = $this->name['value'];
Log::debug(sprintf('Finding budget with name %s', $name));
$filtered = $budgets->filter(
function (Budget $budget) use ($name) {
if ($budget->name === $name) {
Log::debug(sprintf('Found unmapped budget by name (#%d): %s', $budget->id, $budget->name));
return $budget;
}
return null;
}
);
if ($filtered->count() === 1) {
return $filtered->first();
}
Log::debug('Found nothing.');
}
// 4: do not search by account number.
Log::debug('Found NO existing budgets.');
return new Budget;
}
/**
* @return Budget
*/
private function findMappedObject(): Budget
{
Log::debug('In findMappedObject() for Budget');
$fields = ['id', 'name'];
foreach ($fields as $field) {
$array = $this->$field;
Log::debug(sprintf('Find mapped budget based on field "%s" with value', $field), $array);
// check if a pre-mapped object exists.
$mapped = $this->getMappedObject($array);
if (!is_null($mapped->id)) {
Log::debug(sprintf('Found budget #%d!', $mapped->id));
return $mapped;
}
}
Log::debug('Found no budget on mapped data or no map present.');
return new Budget;
}
/**
* @param array $array
*
* @return Budget
*/
private function getMappedObject(array $array): Budget
{
Log::debug('In getMappedObject() for Budget');
if (count($array) === 0) {
Log::debug('Array is empty, nothing will come of this.');
return new Budget;
}
if (array_key_exists('mapped', $array) && is_null($array['mapped'])) {
Log::debug(sprintf('No map present for value "%s". Return NULL.', $array['value']));
return new Budget;
}
Log::debug('Finding a mapped budget based on', $array);
$search = intval($array['mapped']);
$account = $this->repository->find($search);
Log::debug(sprintf('Found budget! #%d ("%s"). Return it', $account->id, $account->name));
return $account;
}
/**
* @return bool
*/
private function store(): bool
{
// 1: find mapped object:
$mapped = $this->findMappedObject();
if (!is_null($mapped->id)) {
$this->budget = $mapped;
return true;
}
// 2: find existing by given values:
$found = $this->findExistingObject();
if (!is_null($found->id)) {
$this->budget = $found;
return true;
}
$name = $this->name['value'] ?? '';
if (strlen($name) === 0) {
return true;
}
Log::debug('Found no budget so must create one ourselves.');
$data = [
'name' => $name,
];
$this->budget = $this->repository->store($data);
Log::debug(sprintf('Successfully stored new budget #%d: %s', $this->budget->id, $this->budget->name));
return true;
}
}

View File

@ -0,0 +1,222 @@
<?php
/**
* ImportCategory.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Models\Category;
use FireflyIII\Repositories\Category\CategoryRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
class ImportCategory
{
/** @var Category */
private $category;
/** @var array */
private $id = [];
/** @var array */
private $name = [];
/** @var CategoryRepositoryInterface */
private $repository;
/** @var User */
private $user;
/**
* @param array $id
*/
public function setId(array $id)
{
$this->id = $id;
}
/**
* ImportCategory constructor.
*/
public function __construct()
{
$this->category = new Category();
$this->repository = app(CategoryRepositoryInterface::class);
Log::debug('Created ImportCategory.');
}
/**
* @param array $name
*/
public function setName(array $name)
{
$this->name = $name;
}
/**
* @return Category
*/
public function getCategory(): Category
{
if (is_null($this->category->id)) {
$this->store();
}
return $this->category;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
$this->repository->setUser($user);
}
/**
* @return Category
*/
private function findExistingObject(): Category
{
Log::debug('In findExistingObject() for Category');
// 1: find by ID, or name
if (count($this->id) === 3) {
Log::debug(sprintf('Finding category with ID #%d', $this->id['value']));
/** @var Category $category */
$category = $this->repository->find(intval($this->id['value']));
if (!is_null($category->id)) {
Log::debug(sprintf('Found unmapped category by ID (#%d): %s', $category->id, $category->name));
return $category;
}
Log::debug('Found nothing.');
}
// 2: find by name
if (count($this->name) === 3) {
/** @var Collection $categories */
$categories = $this->repository->getCategories();
$name = $this->name['value'];
Log::debug(sprintf('Finding category with name %s', $name));
$filtered = $categories->filter(
function (Category $category) use ($name) {
if ($category->name === $name) {
Log::debug(sprintf('Found unmapped category by name (#%d): %s', $category->id, $category->name));
return $category;
}
return null;
}
);
if ($filtered->count() === 1) {
return $filtered->first();
}
Log::debug('Found nothing.');
}
// 4: do not search by account number.
Log::debug('Found NO existing categories.');
return new Category;
}
/**
* @return Category
*/
private function findMappedObject(): Category
{
Log::debug('In findMappedObject() for Category');
$fields = ['id', 'name'];
foreach ($fields as $field) {
$array = $this->$field;
Log::debug(sprintf('Find mapped category based on field "%s" with value', $field), $array);
// check if a pre-mapped object exists.
$mapped = $this->getMappedObject($array);
if (!is_null($mapped->id)) {
Log::debug(sprintf('Found category #%d!', $mapped->id));
return $mapped;
}
}
Log::debug('Found no category on mapped data or no map present.');
return new Category;
}
/**
* @param array $array
*
* @return Category
*/
private function getMappedObject(array $array): Category
{
Log::debug('In getMappedObject() for Category');
if (count($array) === 0) {
Log::debug('Array is empty, nothing will come of this.');
return new Category;
}
if (array_key_exists('mapped', $array) && is_null($array['mapped'])) {
Log::debug(sprintf('No map present for value "%s". Return NULL.', $array['value']));
return new Category;
}
Log::debug('Finding a mapped category based on', $array);
$search = intval($array['mapped']);
$account = $this->repository->find($search);
Log::debug(sprintf('Found category! #%d ("%s"). Return it', $account->id, $account->name));
return $account;
}
/**
* @return bool
*/
private function store(): bool
{
// 1: find mapped object:
$mapped = $this->findMappedObject();
if (!is_null($mapped->id)) {
$this->category = $mapped;
return true;
}
// 2: find existing by given values:
$found = $this->findExistingObject();
if (!is_null($found->id)) {
$this->category = $found;
return true;
}
$name = $this->name['value'] ?? '';
if (strlen($name) === 0) {
return true;
}
Log::debug('Found no category so must create one ourselves.');
$data = [
'name' => $name,
];
$this->category = $this->repository->store($data);
Log::debug(sprintf('Successfully stored new category #%d: %s', $this->category->id, $this->category->name));
return true;
}
}

View File

@ -0,0 +1,218 @@
<?php
/**
* ImportCurrency.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Repositories\Currency\CurrencyRepositoryInterface;
use FireflyIII\User;
use Log;
class ImportCurrency
{
/** @var array */
private $code = [];
/** @var TransactionCurrency */
private $currency;
/** @var array */
private $id = [];
/** @var array */
private $name = [];
/** @var CurrencyRepositoryInterface */
private $repository;
/** @var array */
private $symbol = [];
/** @var User */
private $user;
/**
* ImportCurrency constructor.
*/
public function __construct()
{
$this->currency = new TransactionCurrency;
$this->repository = app(CurrencyRepositoryInterface::class);
}
/**
* @return TransactionCurrency
*/
public function getTransactionCurrency(): TransactionCurrency
{
if (!is_null($this->currency->id)) {
return $this->currency;
}
Log::debug('In createCurrency()');
// check if any of them is mapped:
$mapped = $this->findMappedObject();
if (!is_null($mapped->id)) {
Log::debug('Mapped existing currency.', ['new' => $mapped->toArray()]);
$this->currency = $mapped;
return $mapped;
}
$searched = $this->findExistingObject();
if (!is_null($searched->id)) {
Log::debug('Found existing currency.', ['found' => $searched->toArray()]);
$this->currency = $searched;
return $searched;
}
$data = [
'code' => $this->code['value'] ?? null,
'symbol' => $this->symbol['value'] ?? null,
'name' => $this->name['value'] ?? null,
'decimal_places' => 2,
];
if (is_null($data['code'])) {
Log::info('Need at least a code to create currency, return nothing.');
return new TransactionCurrency();
}
Log::debug('Search for maps resulted in nothing, create new one based on', $data);
$currency = $this->repository->store($data);
$this->currency = $currency;
Log::info('Made new currency.', ['input' => $data, 'new' => $currency->toArray()]);
return $currency;
}
/**
* @param array $code
*/
public function setCode(array $code)
{
$this->code = $code;
}
/**
* @param array $id
*/
public function setId(array $id)
{
$id['value'] = intval($id['value']);
$this->id = $id;
}
/**
* @param array $name
*/
public function setName(array $name)
{
$this->name = $name;
}
/**
* @param array $symbol
*/
public function setSymbol(array $symbol)
{
$this->symbol = $symbol;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
$this->repository->setUser($user);
}
/**
* @return TransactionCurrency
*/
private function findExistingObject(): TransactionCurrency
{
$search = [
'id' => 'find',
'code' => 'findByCode',
'symbol' => 'findBySymbol',
'name' => 'findByName',
];
foreach ($search as $field => $function) {
$value = $this->$field['value'] ?? null;
if (!is_null($value)) {
Log::debug(sprintf('Searching for %s using function %s and value %s', $field, $function, $value));
$currency = $this->repository->$function($value);
if (!is_null($currency->id)) {
return $currency;
}
}
}
return new TransactionCurrency();
}
/**
* @return TransactionCurrency
*/
private function findMappedObject(): TransactionCurrency
{
Log::debug('In findMappedObject()');
$fields = ['id', 'code', 'name', 'symbol'];
foreach ($fields as $field) {
$array = $this->$field;
Log::debug(sprintf('Find mapped currency based on field "%s" with value', $field), $array);
// check if a pre-mapped object exists.
$mapped = $this->getMappedObject($array);
if (!is_null($mapped->id)) {
Log::debug(sprintf('Found currency #%d!', $mapped->id));
return $mapped;
}
}
Log::debug('Found no currency on mapped data or no map present.');
return new TransactionCurrency;
}
/**
* @param array $array
*
* @return TransactionCurrency
*/
private function getMappedObject(array $array): TransactionCurrency
{
Log::debug('In getMappedObject()');
if (count($array) === 0) {
Log::debug('Array is empty, nothing will come of this.');
return new TransactionCurrency;
}
if (array_key_exists('mapped', $array) && is_null($array['mapped'])) {
Log::debug(sprintf('No map present for value "%s". Return NULL.', $array['value']));
return new TransactionCurrency;
}
Log::debug('Finding a mapped object based on', $array);
$search = intval($array['mapped']);
$currency = $this->repository->find($search);
Log::debug(sprintf('Found currency! #%d ("%s"). Return it', $currency->id, $currency->name));
return $currency;
}
}

View File

@ -0,0 +1,265 @@
<?php
/**
* ImportJournal.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use Carbon\Carbon;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Converter\Amount;
use FireflyIII\Import\Converter\ConverterInterface;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Steam;
/**
* Class ImportJournal
*
* @package FireflyIII\Import\Object
*/
class ImportJournal
{
/** @var ImportAccount */
public $asset;
/** @var ImportBudget */
public $budget;
/** @var string */
public $description = '';
/** @var Collection */
public $errors;
/** @var string */
public $hash;
/** @var ImportAccount */
public $opposing;
/** @var string */
private $amount = '0';
/** @var ImportBill */
public $bill;
/** @var ImportCategory */
public $category;
/** @var ImportCurrency */
private $currency;
/** @var string */
private $date = '';
/** @var string */
private $externalId = '';
/** @var array */
private $modifiers = [];
/** @var array */
private $tags = [];
/** @var string */
public $notes = '';
/** @var string */
private $transactionType = '';
/** @var User */
private $user;
/** @var array */
public $metaDates = [];
/**
* ImportEntry constructor.
*/
public function __construct()
{
$this->errors = new Collection;
$this->asset = new ImportAccount;
$this->opposing = new ImportAccount;
$this->bill = new ImportBill;
$this->category = new ImportCategory;
$this->budget = new ImportBudget;
$this->currency = new ImportCurrency;
}
/**
* @param array $modifier
*/
public function addToModifier(array $modifier)
{
$this->modifiers[] = $modifier;
}
/**
* @return TransactionJournal
* @throws FireflyException
*/
public function createTransactionJournal(): TransactionJournal
{
exit('does not work yet');
}
/**
* @return string
*/
public function getAmount(): string
{
/** @var ConverterInterface $amountConverter */
$amountConverter = app(Amount::class);
$this->amount = $amountConverter->convert($this->amount);
// modify
foreach ($this->modifiers as $modifier) {
$class = sprintf('FireflyIII\Import\Converter\%s', config(sprintf('csv.import_roles.%s.converter', $modifier['role'])));
/** @var ConverterInterface $converter */
$converter = app($class);
if ($converter->convert($modifier['value']) === -1) {
$this->amount = Steam::negative($this->amount);
}
}
return $this->amount;
}
/**
* @return ImportCurrency
*/
public function getCurrency(): ImportCurrency
{
return $this->currency;
}
/**
* @param string $format
*
* @return Carbon
*/
public function getDate(string $format): Carbon
{
return Carbon::createFromFormat($format, $this->date);
}
/**
* @param string $hash
*/
public function setHash(string $hash)
{
$this->hash = $hash;
}
/**
* @param User $user
*/
public function setUser(User $user)
{
$this->user = $user;
// set user for related objects:
$this->asset->setUser($user);
$this->opposing->setUser($user);
$this->budget->setUser($user);
$this->category->setUser($user);
$this->bill->setUser($user);
}
/**
* @param array $array
*
* @throws FireflyException
*/
public function setValue(array $array)
{
switch ($array['role']) {
default:
throw new FireflyException(sprintf('ImportJournal cannot handle "%s" with value "%s".', $array['role'], $array['value']));
case 'account-id':
$this->asset->setAccountId($array);
break;
case 'amount':
$this->amount = $array['value'];
break;
case 'account-iban':
$this->asset->setAccountIban($array);
break;
case 'account-name':
$this->asset->setAccountName($array);
break;
case 'account-number':
$this->asset->setAccountNumber($array);
break;
case 'bill-id':
$this->bill->setId($array);
break;
case 'bill-name':
$this->bill->setName($array);
break;
case 'budget-id':
$this->budget->setId($array);
break;
case 'budget-name':
$this->budget->setName($array);
break;
case 'category-id':
$this->category->setId($array);
break;
case 'category-name':
$this->category->setName($array);
break;
case 'currency-code':
$this->currency->setCode($array);
break;
case 'currency-id':
$this->currency->setId($array);
break;
case 'currency-name':
$this->currency->setName($array);
break;
case 'currency-symbol':
$this->currency->setSymbol($array);
break;
case 'date-transaction':
$this->date = $array['value'];
break;
case 'description':
$this->description = $array['value'];
break;
case 'sepa-ct-op':
case 'sepa-ct-id':
case 'sepa-db':
$this->notes .= ' '.$array['value'];
$this->notes = trim($this->notes);
break;
case 'external-id':
$this->externalId = $array['value'];
break;
case '_ignore':
break;
case 'ing-debet-credit':
case 'rabo-debet-credit':
$this->addToModifier($array);
break;
case 'opposing-iban':
$this->opposing->setAccountIban($array);
break;
case 'opposing-name':
$this->opposing->setAccountName($array);
break;
case 'opposing-number':
$this->opposing->setAccountNumber($array);
break;
case 'opposing-id':
$this->opposing->setAccountId($array);
break;
case 'tags-comma':
case 'tags-space':
$this->tags[] = $array;
break;
case 'date-interest':
$this->metaDates['interest_date'] = $array['value'];
break;
case 'date-book':
$this->metaDates['book_date'] = $array['value'];
break;
case 'date-process':
$this->metaDates['process_date'] = $array['value'];
break;
}
}
}

View File

@ -0,0 +1,157 @@
<?php
/**
* ImportTransaction.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Object;
use FireflyIII\Import\Converter\ConverterInterface;
use Steam;
class ImportTransaction
{
/** @var string */
private $amount;
/** @var ImportCurrency */
private $currency;
/** @var string */
private $date;
/** @var string */
private $description;
private $modifiers = [];
/** @var bool */
private $positive = true;
public function __construct()
{
$this->currency = new ImportCurrency;
}
public function addToModifier(array $modifier)
{
$this->modifiers[] = $modifier;
}
/**
* @return string
*/
public function getAmount(): string
{
// use converter:
$this->amount = strval($this->parseAmount());
// also apply modifiers:
$this->amount = Steam::positive($this->amount);
// Can handle ING
foreach ($this->modifiers as $modifier) {
$class = sprintf('FireflyIII\Import\Converter\%s', config(sprintf('csv.import_roles.%s.converter', $modifier['role'])));
/** @var ConverterInterface $converter */
$converter = app($class);
if ($converter->convert($modifier['value']) === -1) {
$this->amount = Steam::negative($this->amount);
}
}
return $this->amount;
}
/**
* @param string $amount
*/
public function setAmount(string $amount)
{
$this->amount = $amount;
}
/**
* @return ImportCurrency
*/
public function getCurrency(): ImportCurrency
{
return $this->currency;
}
/**
* @param ImportCurrency $currency
*/
public function setCurrency(ImportCurrency $currency)
{
$this->currency = $currency;
}
/**
* @param string $date
*/
public function setDate(string $date)
{
$this->date = $date;
}
/**
* @param string $description
*/
public function setDescription(string $description)
{
$this->description = $description;
}
/**
* @param bool $positive
*/
public function setPositive(bool $positive)
{
$this->positive = $positive;
}
/**
* Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
* - Jamie Zawinski
*
* @return float
*/
private function parseAmount()
{
$value = $this->amount;
$len = strlen($value);
$decimalPosition = $len - 3;
$decimal = null;
if (($len > 2 && $value{$decimalPosition} == '.') || ($len > 2 && strpos($value, '.') > $decimalPosition)) {
$decimal = '.';
}
if ($len > 2 && $value{$decimalPosition} == ',') {
$decimal = ',';
}
// if decimal is dot, replace all comma's and spaces with nothing. then parse as float (round to 4 pos)
if ($decimal === '.') {
$search = [',', ' '];
$value = str_replace($search, '', $value);
}
if ($decimal === ',') {
$search = ['.', ' '];
$value = str_replace($search, '', $value);
$value = str_replace(',', '.', $value);
}
if (is_null($decimal)) {
// replace all:
$search = ['.', ' ', ','];
$value = str_replace($search, '', $value);
}
return round(floatval($value), 12);
}
}

View File

@ -0,0 +1,166 @@
<?php
/**
* ImportRoutine.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Routine;
use Carbon\Carbon;
use FireflyIII\Import\FileProcessor\FileProcessorInterface;
use FireflyIII\Import\Storage\ImportStorage;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\Tag;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
use Illuminate\Support\Collection;
use Log;
class ImportRoutine
{
/** @var Collection */
public $errors;
/** @var Collection */
public $journals;
/** @var int */
public $lines = 0;
/** @var ImportJob */
private $job;
/**
* ImportRoutine constructor.
*
*/
public function __construct()
{
$this->journals = new Collection;
$this->errors = new Collection;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
}
/**
*
*/
public function run(): bool
{
if ($this->job->status !== 'configured') {
Log::error(sprintf('Job %s is in state "%s" so it cannot be started.', $this->job->key, $this->job->status));
return false;
}
set_time_limit(0);
Log::info(sprintf('Start with import job %s', $this->job->key));
$importObjects = $this->getImportObjects();
$this->lines = $importObjects->count();
// once done, use storage thing to actually store them:
Log::info(sprintf('Returned %d valid objects from file processor', $this->lines));
$storage = $this->storeObjects($importObjects);
// update job:
$this->job->status = 'finished';
$this->job->save();
$this->journals = $storage->journals;
$this->errors = $storage->errors;
// create tag, link tag to all journals:
$this->createImportTag();
Log::info(sprintf('Done with import job %s', $this->job->key));
return true;
}
/**
* @return Collection
*/
protected function getImportObjects(): Collection
{
$objects = new Collection;
$type = $this->job->file_type;
$class = config(sprintf('firefly.import_processors.%s', $type));
/** @var FileProcessorInterface $processor */
$processor = app($class);
$processor->setJob($this->job);
if ($this->job->status == 'configured') {
// set job as "running"...
$this->job->status = 'running';
$this->job->save();
Log::debug('Job is configured, start with run()');
$processor->run();
$objects = $processor->getObjects();
}
return $objects;
}
/**
*
*/
private function createImportTag(): Tag
{
/** @var TagRepositoryInterface $repository */
$repository = app(TagRepositoryInterface::class);
$repository->setUser($this->job->user);
$data = [
'tag' => trans('firefly.import_with_key', ['key' => $this->job->key]),
'date' => new Carbon,
'description' => null,
'latitude' => null,
'longitude' => null,
'zoomLevel' => null,
'tagMode' => 'nothing',
];
$tag = $repository->store($data);
$extended = $this->job->extended_status;
$extended['tag'] = $tag->id;
$this->job->extended_status = $extended;
$this->job->save();
$this->journals->each(
function (TransactionJournal $journal) use ($tag) {
$journal->tags()->save($tag);
}
);
return $tag;
}
/**
* @param Collection $objects
*
* @return ImportStorage
*/
private function storeObjects(Collection $objects): ImportStorage
{
$storage = new ImportStorage;
$storage->setJob($this->job);
$storage->setDateFormat($this->job->configuration['date-format']);
$storage->setObjects($objects);
$storage->store();
return $storage;
}
}

View File

@ -1,287 +0,0 @@
<?php
/**
* CsvSetup.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Setup;
use ExpandedForm;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Models\ImportJob;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use FireflyIII\Support\Import\CsvImportSupportTrait;
use Illuminate\Http\Request;
use Log;
use Symfony\Component\HttpFoundation\FileBag;
/**
* Class CsvSetup
*
* @package FireflyIII\Import\Importer
*/
class CsvSetup implements SetupInterface
{
use CsvImportSupportTrait;
/** @var Account */
public $defaultImportAccount;
/** @var ImportJob */
public $job;
/**
* CsvImporter constructor.
*/
public function __construct()
{
$this->defaultImportAccount = new Account;
}
/**
* Create initial (empty) configuration array.
*
* @return bool
*/
public function configure(): bool
{
if (is_null($this->job->configuration) || (is_array($this->job->configuration) && count($this->job->configuration) === 0)) {
Log::debug('No config detected, will create empty one.');
$this->job->configuration = config('csv.default_config');
$this->job->save();
return true;
}
// need to do nothing, for now.
Log::debug('Detected config in upload, will use that one. ', $this->job->configuration);
return true;
}
/**
* @return array
*/
public function getConfigurationData(): array
{
/** @var AccountRepositoryInterface $accountRepository */
$accountRepository = app(AccountRepositoryInterface::class);
$accounts = $accountRepository->getAccountsByType([AccountType::DEFAULT, AccountType::ASSET]);
$delimiters = [
',' => trans('form.csv_comma'),
';' => trans('form.csv_semicolon'),
'tab' => trans('form.csv_tab'),
];
$specifics = [];
// collect specifics.
foreach (config('csv.import_specifics') as $name => $className) {
$specifics[$name] = [
'name' => $className::getName(),
'description' => $className::getDescription(),
];
}
$data = [
'accounts' => ExpandedForm::makeSelectList($accounts),
'specifix' => [],
'delimiters' => $delimiters,
'upload_path' => storage_path('upload'),
'is_upload_possible' => is_writable(storage_path('upload')),
'specifics' => $specifics,
];
return $data;
}
/**
* This method returns the data required for the view that will let the user add settings to the import job.
*
* @return array
*/
public function getDataForSettings(): array
{
Log::debug('Now in getDataForSettings()');
if ($this->doColumnRoles()) {
Log::debug('doColumnRoles() is true.');
$data = $this->getDataForColumnRoles();
return $data;
}
if ($this->doColumnMapping()) {
Log::debug('doColumnMapping() is true.');
$data = $this->getDataForColumnMapping();
return $data;
}
echo 'no settings to do.';
exit;
}
/**
* This method returns the name of the view that will be shown to the user to further configure
* the import job.
*
* @return string
* @throws FireflyException
*/
public function getViewForSettings(): string
{
if ($this->doColumnRoles()) {
return 'import.csv.roles';
}
if ($this->doColumnMapping()) {
return 'import.csv.map';
}
throw new FireflyException('There is no view for the current CSV import step.');
}
/**
* This method returns whether or not the user must configure this import
* job further.
*
* @return bool
*/
public function requireUserSettings(): bool
{
Log::debug(sprintf('doColumnMapping is %s', $this->doColumnMapping()));
Log::debug(sprintf('doColumnRoles is %s', $this->doColumnRoles()));
if ($this->doColumnMapping() || $this->doColumnRoles()) {
Log::debug('Return true');
return true;
}
Log::debug('Return false');
return false;
}
/**
* @param array $data
*
* @param FileBag $files
*
* @return bool
*/
public function saveImportConfiguration(array $data, FileBag $files): bool
{
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$importId = $data['csv_import_account'] ?? 0;
$account = $repository->find(intval($importId));
$hasHeaders = isset($data['has_headers']) && intval($data['has_headers']) === 1 ? true : false;
$config = $this->job->configuration;
$config['has-headers'] = $hasHeaders;
$config['date-format'] = $data['date_format'];
$config['delimiter'] = $data['csv_delimiter'];
$config['delimiter'] = $config['delimiter'] === 'tab' ? "\t" : $config['delimiter'];
Log::debug('Entered import account.', ['id' => $importId]);
if (!is_null($account->id)) {
Log::debug('Found account.', ['id' => $account->id, 'name' => $account->name]);
$config['import-account'] = $account->id;
} else {
Log::error('Could not find anything for csv_import_account.', ['id' => $importId]);
}
// loop specifics.
if (isset($data['specifics']) && is_array($data['specifics'])) {
foreach ($data['specifics'] as $name => $enabled) {
// verify their content.
$className = sprintf('FireflyIII\Import\Specifics\%s', $name);
if (class_exists($className)) {
$config['specifics'][$name] = 1;
}
}
}
$this->job->configuration = $config;
$this->job->save();
return true;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
}
/**
* Store the settings filled in by the user, if applicable.
*
* @param Request $request
*
*/
public function storeSettings(Request $request)
{
$config = $this->job->configuration;
$all = $request->all();
if ($request->get('settings') == 'roles') {
$count = $config['column-count'];
$roleSet = 0; // how many roles have been defined
$mapSet = 0; // how many columns must be mapped
for ($i = 0; $i < $count; $i++) {
$selectedRole = $all['role'][$i] ?? '_ignore';
$doMapping = isset($all['map'][$i]) && $all['map'][$i] == '1' ? true : false;
if ($selectedRole == '_ignore' && $doMapping === true) {
$doMapping = false; // cannot map ignored columns.
}
if ($selectedRole != '_ignore') {
$roleSet++;
}
if ($doMapping === true) {
$mapSet++;
}
$config['column-roles'][$i] = $selectedRole;
$config['column-do-mapping'][$i] = $doMapping;
}
if ($roleSet > 0) {
$config['column-roles-complete'] = true;
$this->job->configuration = $config;
$this->job->save();
}
if ($mapSet === 0) {
// skip setting of map:
$config['column-mapping-complete'] = true;
}
}
if ($request->get('settings') == 'map') {
if (isset($all['mapping'])) {
foreach ($all['mapping'] as $index => $data) {
$config['column-mapping-config'][$index] = [];
foreach ($data as $value => $mapId) {
$mapId = intval($mapId);
if ($mapId !== 0) {
$config['column-mapping-config'][$index][$value] = intval($mapId);
}
}
}
}
// set thing to be completed.
$config['column-mapping-complete'] = true;
$this->job->configuration = $config;
$this->job->save();
}
}
}

View File

@ -1,88 +0,0 @@
<?php
/**
* SetupInterface.php
* Copyright (C) 2016 thegrumpydictator@gmail.com
*
* This software may be modified and distributed under the terms of the
* Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Setup;
use FireflyIII\Models\ImportJob;
use Illuminate\Http\Request;
use Symfony\Component\HttpFoundation\FileBag;
/**
* Interface SetupInterface
*
* @package FireflyIII\Import\Setup
*/
interface SetupInterface
{
/**
* After uploading, and after setJob(), prepare anything that is
* necessary for the configure() line.
*
* @return bool
*/
public function configure(): bool;
/**
* Returns any data necessary to do the configuration.
*
* @return array
*/
public function getConfigurationData(): array;
/**
* This method returns the data required for the view that will let the user add settings to the import job.
*
* @return array
*/
public function getDataForSettings(): array;
/**
* This method returns the name of the view that will be shown to the user to further configure
* the import job.
*
* @return string
*/
public function getViewForSettings(): string;
/**
* This method returns whether or not the user must configure this import
* job further.
*
* @return bool
*/
public function requireUserSettings(): bool;
/**
* @param array $data
*
* @param FileBag $files
*
* @return bool
*/
public function saveImportConfiguration(array $data, FileBag $files): bool;
/**
* @param ImportJob $job
*
*/
public function setJob(ImportJob $job);
/**
* Store the settings filled in by the user, if applicable.
*
* @param Request $request
*
*/
public function storeSettings(Request $request);
}

View File

@ -0,0 +1,376 @@
<?php
/**
* ImportStorage.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Import\Storage;
use Amount;
use Carbon\Carbon;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Object\ImportAccount;
use FireflyIII\Import\Object\ImportJournal;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\Models\Bill;
use FireflyIII\Models\Budget;
use FireflyIII\Models\Category;
use FireflyIII\Models\ImportJob;
use FireflyIII\Models\Rule;
use FireflyIII\Models\Transaction;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\Models\TransactionType;
use FireflyIII\Rules\Processor;
use Illuminate\Support\Collection;
use Illuminate\Support\MessageBag;
use Log;
use Steam;
/**
* Is capable of storing individual ImportJournal objects.
* Class ImportStorage
*
* @package FireflyIII\Import\Storage
*/
class ImportStorage
{
/** @var Collection */
public $errors;
public $journals;
/** @var string */
private $dateFormat = 'Ymd';
/** @var TransactionCurrency */
private $defaultCurrency;
/** @var ImportJob */
private $job;
/** @var Collection */
private $objects;
/** @var Collection */
private $rules;
/**
* ImportStorage constructor.
*/
public function __construct()
{
$this->objects = new Collection;
$this->journals = new Collection;
$this->errors = new Collection;
}
/**
* @param string $dateFormat
*/
public function setDateFormat(string $dateFormat)
{
$this->dateFormat = $dateFormat;
}
/**
* @param ImportJob $job
*/
public function setJob(ImportJob $job)
{
$this->job = $job;
$this->rules = $this->getUserRules();
}
/**
* @param Collection $objects
*/
public function setObjects(Collection $objects)
{
$this->objects = $objects;
}
/**
* Do storage of import objects
*/
public function store()
{
$this->defaultCurrency = Amount::getDefaultCurrencyByUser($this->job->user);
// routine below consists of 3 steps.
/**
* @var int $index
* @var ImportJournal $object
*/
foreach ($this->objects as $index => $object) {
try {
$this->storeImportJournal($index, $object);
} catch (FireflyException $e) {
$this->errors->push($e->getMessage());
}
}
}
/**
* @param TransactionJournal $journal
*
* @return bool
*/
protected function applyRules(TransactionJournal $journal): bool
{
if ($this->rules->count() > 0) {
/** @var Rule $rule */
foreach ($this->rules as $rule) {
Log::debug(sprintf('Going to apply rule #%d to journal %d.', $rule->id, $journal->id));
$processor = Processor::make($rule);
$processor->handleTransactionJournal($journal);
if ($rule->stop_processing) {
return true;
}
}
}
return true;
}
/**
* @param int $journalId
* @param int $accountId
* @param int $currencyId
* @param string $amount
*
* @return bool
* @throws FireflyException
*/
private function createTransaction(int $journalId, int $accountId, int $currencyId, string $amount): bool
{
$transaction = new Transaction;
$transaction->account_id = $accountId;
$transaction->transaction_journal_id = $journalId;
$transaction->transaction_currency_id = $currencyId;
$transaction->amount = $amount;
$transaction->save();
if (is_null($transaction->id)) {
$errorText = join(', ', $transaction->getErrors()->all());
throw new FireflyException($errorText);
}
Log::debug(sprintf('Created transaction with ID #%d and account #%d', $transaction->id, $accountId));
return true;
}
/**
* @param ImportJournal $importJournal
*
* @return TransactionCurrency
*/
private function getCurrency(ImportJournal $importJournal): TransactionCurrency
{
$currency = $importJournal->getCurrency()->getTransactionCurrency();
if (is_null($currency->id)) {
$currency = $this->defaultCurrency;
}
return $currency;
}
/**
* @param ImportAccount $account
* @param $amount
*
* @return Account
*/
private function getOpposingAccount(ImportAccount $account, $amount): Account
{
if (bccomp($amount, '0') === -1) {
Log::debug(sprintf('%s is negative, create opposing expense account.', $amount));
$account->setExpectedType(AccountType::EXPENSE);
return $account->getAccount();
}
Log::debug(sprintf('%s is positive, create opposing revenue account.', $amount));
// amount is positive, it's a deposit, opposing is an revenue:
$account->setExpectedType(AccountType::REVENUE);
return $account->getAccount();
}
/**
* @param string $amount
*
* @return TransactionType
*/
private function getTransactionType(string $amount): TransactionType
{
$transactionType = new TransactionType();
// amount is negative, it's a withdrawal, opposing is an expense:
if (bccomp($amount, '0') === -1) {
$transactionType = TransactionType::whereType(TransactionType::WITHDRAWAL)->first();
}
if (bccomp($amount, '0') === 1) {
$transactionType = TransactionType::whereType(TransactionType::DEPOSIT)->first();
}
return $transactionType;
}
/**
* @return Collection
*/
private function getUserRules(): Collection
{
$set = Rule::distinct()
->where('rules.user_id', $this->job->user->id)
->leftJoin('rule_groups', 'rule_groups.id', '=', 'rules.rule_group_id')
->leftJoin('rule_triggers', 'rules.id', '=', 'rule_triggers.rule_id')
->where('rule_groups.active', 1)
->where('rule_triggers.trigger_type', 'user_action')
->where('rule_triggers.trigger_value', 'store-journal')
->where('rules.active', 1)
->orderBy('rule_groups.order', 'ASC')
->orderBy('rules.order', 'ASC')
->get(['rules.*', 'rule_groups.order']);
Log::debug(sprintf('Found %d user rules.', $set->count()));
return $set;
}
/**
* @param TransactionJournal $journal
* @param Bill $bill
*/
private function storeBill(TransactionJournal $journal, Bill $bill)
{
if (!is_null($bill->id)) {
Log::debug(sprintf('Linked bill #%d to journal #%d', $bill->id, $journal->id));
$journal->bill()->associate($bill);
$journal->save();
}
}
/**
* @param TransactionJournal $journal
* @param Budget $budget
*/
private function storeBudget(TransactionJournal $journal, Budget $budget)
{
if (!is_null($budget->id)) {
Log::debug(sprintf('Linked budget #%d to journal #%d', $budget->id, $journal->id));
$journal->budgets()->save($budget);
}
}
/**
* @param TransactionJournal $journal
* @param Category $category
*/
private function storeCategory(TransactionJournal $journal, Category $category)
{
if (!is_null($category->id)) {
Log::debug(sprintf('Linked category #%d to journal #%d', $category->id, $journal->id));
$journal->categories()->save($category);
}
}
private function storeImportJournal(int $index, ImportJournal $importJournal): bool
{
Log::debug(sprintf('Going to store object #%d with description "%s"', $index, $importJournal->description));
$asset = $importJournal->asset->getAccount();
$amount = $importJournal->getAmount();
$currency = $this->getCurrency($importJournal);
$date = $importJournal->getDate($this->dateFormat);
$transactionType = $this->getTransactionType($amount);
$opposing = $this->getOpposingAccount($importJournal->opposing, $amount);
// if opposing is an asset account, it's a transfer:
if ($opposing->accountType->type === AccountType::ASSET) {
Log::debug(sprintf('Opposing account #%d %s is an asset account, make transfer.', $opposing->id, $opposing->name));
$transactionType = TransactionType::whereType(TransactionType::TRANSFER)->first();
}
/*** First step done! */
$this->job->addStepsDone(1);
// create a journal:
$journal = new TransactionJournal;
$journal->user_id = $this->job->user_id;
$journal->transaction_type_id = $transactionType->id;
$journal->transaction_currency_id = $currency->id;
$journal->description = $importJournal->description;
$journal->date = $date->format('Y-m-d');
$journal->order = 0;
$journal->tag_count = 0;
$journal->encrypted = 0;
$journal->completed = false;
if (!$journal->save()) {
$errorText = join(', ', $journal->getErrors()->all());
$this->job->addStepsDone(3);
throw new FireflyException($errorText);
}
// save meta data:
$journal->setMeta('importHash', $importJournal->hash);
Log::debug(sprintf('Created journal with ID #%d', $journal->id));
// create transactions:
$this->createTransaction($journal->id, $asset->id, $currency->id, $amount);
$this->createTransaction($journal->id, $opposing->id, $currency->id, Steam::opposite($amount));
/*** Another step done! */
$this->job->addStepsDone(1);
// store meta object things:
$this->storeCategory($journal, $importJournal->category->getCategory());
$this->storeBudget($journal, $importJournal->budget->getBudget());
$this->storeBill($journal, $importJournal->bill->getBill());
$this->storeMeta($journal, $importJournal->metaDates);
// sepa thing as note:
if (strlen($importJournal->notes) > 0) {
$journal->setMeta('notes', $importJournal->notes);
}
// set journal completed:
$journal->completed = true;
$journal->save();
$this->job->addStepsDone(1);
// run rules:
$this->applyRules($journal);
$this->job->addStepsDone(1);
$this->journals->push($journal);
return true;
}
/**
* @param TransactionJournal $journal
* @param array $dates
*/
private function storeMeta(TransactionJournal $journal, array $dates)
{
// all other date fields as meta thing:
foreach ($dates as $name => $value) {
try {
$date = new Carbon($value);
$journal->setMeta($name, $date);
} catch (\Exception $e) {
// don't care, ignore:
Log::warning(sprintf('Could not parse "%s" into a valid Date object for field %s', $value, $name));
}
}
}
}

View File

@ -1,47 +0,0 @@
The import routine is as follows:
1. Upload and setup:
User uploads a file with entries. The Setup/SetupInterface gives the user
the opportunity (in any number of steps) to do the necessary configuration.
This could also be skipped of course. An ImportJob object is created with a
basic and empty configuration.
Helper classes are as follows, greatly modelled to the CSV importer:
- The Mapper classes give back lists of Firefly objects. You can show them to the
user in order to help you convert text values to their Firefly counterparts.
For example, the user maps "Gcrsr" to category Groceries.
- The Converter classes exist to help convert text values to their Firely counterparts.
Feed "AB12ABCD897829" to the AssetAccountIban Converter and you should end up with a new
or found asset account. The previously built mapping is used to narrow it down. Submit an empty
mapping if this one is not relevant.
The mapping and possibly other configuration options are stored in a newly created
ImportJob object, stored in the database. This import job holds a reference to the uploaded file
(placed encrypted in /storage/uploads) and the status of the import.
2. Actual import
Using either the command line or the web interface the user can tell Firefly to start the import.
The ImporterInterface runs and creates an ImportEntry for each line, blob or whatever distinction it
wants.
For each line, the ImporterInterface should run each field through the selected Converter in order
to convert the text values to their Firefly counterparts. Again, this is modelled to the CSV importer
and may need an update for MT940.
In any case, this newly minted set of ImportEntries (it cannot be saved or stored atm,
this has to be done in one go) is then fed to the ImportValidator which will reject entries
(almost never) and corrects fields if necessary.
- Adds the date (today) if no date is present.
- Determins the type of transaction (withdrawal, deposit, transfer).
- Determins the types of accounts involved (asset, expense, revenue).
- Determins the currency of the transaction.
- Adds a default description if there isn't one present.
This set of corrected ImportEntries is then fed to the ImportStorage class which will generate
TransactionJournals, Transactions and other related objects.

View File

@ -42,11 +42,11 @@ class ImportJob extends Model
protected $validStatus
= [
'import_status_never_started', // initial state
'import_configuration_saved', // import configuration saved. This step is going to be obsolete.
'settings_complete', // aka: ready for import.
'import_running', // import currently underway
'import_complete', // done with everything
'new',
'initialized',
'configured',
'running',
'finished',
];
/**
@ -66,13 +66,28 @@ class ImportJob extends Model
throw new NotFoundHttpException;
}
/**
* @param int $index
* @param string $message
*
* @return bool
*/
public function addError(int $index, string $message): bool
{
$extended = $this->extended_status;
$extended['errors'][$index][] = $message;
$this->extended_status = $extended;
return true;
}
/**
* @param int $count
*/
public function addStepsDone(int $count)
{
$status = $this->extended_status;
$status['steps_done'] += $count;
$status['done'] += $count;
$this->extended_status = $status;
$this->save();
@ -84,7 +99,7 @@ class ImportJob extends Model
public function addTotalSteps(int $count)
{
$status = $this->extended_status;
$status['total_steps'] += $count;
$status['steps'] += $count;
$this->extended_status = $status;
$this->save();
@ -165,7 +180,7 @@ class ImportJob extends Model
$disk = Storage::disk('upload');
$encryptedContent = $disk->get($fileName);
$content = Crypt::decrypt($encryptedContent);
Log::debug(sprintf('Content size is %d bytes.', $content));
Log::debug(sprintf('Content size is %d bytes.', strlen($content)));
return $content;
}

View File

@ -133,6 +133,9 @@ class Tag extends Model
*/
public function getTagAttribute($value)
{
if(is_null($value)) {
return null;
}
return Crypt::decrypt($value);
}

View File

@ -243,9 +243,11 @@ class AccountRepository implements AccountRepositoryInterface
}
// account may exist already:
$existingAccount = $this->findByName($data['name'], [$data['accountType']]);
$existingAccount = $this->findByName($data['name'], [$type]);
if (!is_null($existingAccount->id)) {
throw new FireflyException(sprintf('There already is an account named "%s" of type "%s".', $data['name'], $data['accountType']));
Log::warning(sprintf('There already is an account named "%s" of type "%s".', $data['name'], $type));
return $existingAccount;
}
// create it:
@ -267,6 +269,7 @@ class AccountRepository implements AccountRepositoryInterface
);
throw new FireflyException(sprintf('Tried to create account named "%s" but failed. The logs have more details.', $data['name']));
}
Log::debug(sprintf('Created new account #%d named "%s" of type %s.', $newAccount->id, $newAccount->name, $accountType->type));
return $newAccount;
}

View File

@ -13,11 +13,14 @@ namespace FireflyIII\Repositories\Account;
use FireflyIII\Models\Account;
use FireflyIII\Models\AccountType;
use FireflyIII\User;
use Illuminate\Database\Eloquent\Relations\HasMany;
use Illuminate\Support\Collection;
use Log;
/**
* @property User $user
*
* Trait FindAccountsTrait
*
* @package FireflyIII\Repositories\Account
@ -107,7 +110,7 @@ trait FindAccountsTrait
$query->whereIn('account_types.type', $types);
}
Log::debug(sprintf('Searching for account named %s of the following type(s)', $name), ['types' => $types]);
Log::debug(sprintf('Searching for account named "%s" (of user #%d) of the following type(s)', $name, $this->user->id), ['types' => $types]);
$accounts = $query->get(['accounts.*']);
/** @var Account $account */
@ -118,7 +121,7 @@ trait FindAccountsTrait
return $account;
}
}
Log::debug('Found nothing.');
Log::debug(sprintf('There is no account with name "%s" or types', $name), $types);
return new Account;
}

View File

@ -19,6 +19,7 @@ use FireflyIII\Helpers\Attachments\AttachmentHelperInterface;
use FireflyIII\Models\Attachment;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
use Storage;
/**
@ -97,16 +98,20 @@ class AttachmentRepository implements AttachmentRepositoryInterface
// create a disk.
$disk = Storage::disk('upload');
$file = $attachment->fileName();
$content = '';
if ($disk->exists($file)) {
$content = Crypt::decrypt($disk->get($file));
return $content;
}
if (is_bool($content)) {
Log::error(sprintf('Attachment #%d may be corrupted: the content could not be decrypted.', $attachment->id));
return '';
}
return $content;
}
/**
* @param User $user
*/

View File

@ -262,6 +262,8 @@ class BillRepository implements BillRepositoryInterface
}
/**
* The "paid dates" list is a list of dates of transaction journals that are linked to this bill.
*
* @param Bill $bill
* @param Carbon $start
* @param Carbon $end

View File

@ -13,10 +13,16 @@ declare(strict_types=1);
namespace FireflyIII\Repositories\ImportJob;
use Crypt;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Models\ImportJob;
use FireflyIII\Repositories\User\UserRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Str;
use Log;
use SplFileObject;
use Storage;
use Symfony\Component\HttpFoundation\File\UploadedFile;
/**
* Class ImportJobRepository
@ -51,12 +57,12 @@ class ImportJobRepository implements ImportJobRepositoryInterface
$importJob->user()->associate($this->user);
$importJob->file_type = $fileType;
$importJob->key = Str::random(12);
$importJob->status = 'import_status_never_started';
$importJob->status = 'new';
$importJob->configuration = [];
$importJob->extended_status = [
'total_steps' => 0,
'steps_done' => 0,
'import_count' => 0,
'importTag' => 0,
'steps' => 0,
'done' => 0,
'tag' => 0,
'errors' => [],
];
$importJob->save();
@ -86,6 +92,75 @@ class ImportJobRepository implements ImportJobRepositoryInterface
return $result;
}
/**
* @param ImportJob $job
* @param UploadedFile $file
*
* @return bool
*/
public function processConfiguration(ImportJob $job, UploadedFile $file): bool
{
/** @var UserRepositoryInterface $repository */
$repository = app(UserRepositoryInterface::class);
// demo user's configuration upload is ignored completely.
if (!$repository->hasRole($this->user, 'demo')) {
Log::debug(
'Uploaded configuration file', ['name' => $file->getClientOriginalName(), 'size' => $file->getSize(), 'mime' => $file->getClientMimeType()]
);
$configFileObject = new SplFileObject($file->getRealPath());
$configRaw = $configFileObject->fread($configFileObject->getSize());
$configuration = json_decode($configRaw, true);
if (!is_null($configuration) && is_array($configuration)) {
Log::debug('Found configuration', $configuration);
$this->setConfiguration($job, $configuration);
}
}
return true;
}
/**
* @param ImportJob $job
* @param UploadedFile $file
*
* @return mixed
*/
public function processFile(ImportJob $job, UploadedFile $file): bool
{
/** @var UserRepositoryInterface $repository */
$repository = app(UserRepositoryInterface::class);
$newName = sprintf('%s.upload', $job->key);
$uploaded = new SplFileObject($file->getRealPath());
$content = $uploaded->fread($uploaded->getSize());
$contentEncrypted = Crypt::encrypt($content);
$disk = Storage::disk('upload');
// user is demo user, replace upload with prepared file.
if ($repository->hasRole($this->user, 'demo')) {
$stubsDisk = Storage::disk('stubs');
$content = $stubsDisk->get('demo-import.csv');
$contentEncrypted = Crypt::encrypt($content);
$disk->put($newName, $contentEncrypted);
Log::debug('Replaced upload with demo file.');
// also set up prepared configuration.
$configuration = json_decode($stubsDisk->get('demo-configuration.json'), true);
$this->setConfiguration($job, $configuration);
Log::debug('Set configuration for demo user', $configuration);
}
if (!$repository->hasRole($this->user, 'demo')) {
// user is not demo, process original upload:
$disk->put($newName, $contentEncrypted);
Log::debug('Uploaded file', ['name' => $file->getClientOriginalName(), 'size' => $file->getSize(), 'mime' => $file->getClientMimeType()]);
}
return true;
}
/**
* @param ImportJob $job
* @param array $configuration

View File

@ -15,6 +15,7 @@ namespace FireflyIII\Repositories\ImportJob;
use FireflyIII\Models\ImportJob;
use FireflyIII\User;
use Symfony\Component\HttpFoundation\File\UploadedFile;
/**
* Interface ImportJobRepositoryInterface
@ -37,6 +38,22 @@ interface ImportJobRepositoryInterface
*/
public function findByKey(string $key): ImportJob;
/**
* @param ImportJob $job
* @param UploadedFile $file
*
* @return mixed
*/
public function processFile(ImportJob $job, UploadedFile $file): bool;
/**
* @param ImportJob $job
* @param UploadedFile $file
*
* @return bool
*/
public function processConfiguration(ImportJob $job, UploadedFile $file): bool;
/**
* @param ImportJob $job
* @param array $configuration

View File

@ -19,16 +19,21 @@ use FireflyIII\Models\Transaction;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\Models\TransactionType;
use FireflyIII\Repositories\Tag\TagRepositoryInterface;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Log;
/**
* @property User $user
*
* Trait CreateJournalsTrait
*
* @package FireflyIII\Repositories\Journal
*/
trait CreateJournalsTrait
{
/**
*
* * Remember: a balancingAct takes at most one expense and one transfer.

View File

@ -18,6 +18,7 @@ use FireflyIII\Models\Transaction as TransactionModel;
use FireflyIII\Models\TransactionCurrency;
use FireflyIII\Models\TransactionJournal;
use FireflyIII\Models\TransactionType;
use FireflyIII\User;
use Illuminate\Support\Collection;
use Preferences as Prefs;
@ -208,13 +209,27 @@ class Amount
* @throws FireflyException
*/
public function getDefaultCurrency(): TransactionCurrency
{
$user = auth()->user();
return $this->getDefaultCurrencyByUser($user);
}
/**
* @param User $user
*
* @return \FireflyIII\Models\TransactionCurrency
* @throws FireflyException
*/
public function getDefaultCurrencyByUser(User $user): TransactionCurrency
{
$cache = new CacheProperties;
$cache->addProperty('getDefaultCurrency');
$cache->addProperty($user->id);
if ($cache->has()) {
return $cache->get(); // @codeCoverageIgnore
}
$currencyPreference = Prefs::get('currencyPreference', config('firefly.default_currency', 'EUR'));
$currencyPreference = Prefs::getForUser($user, 'currencyPreference', config('firefly.default_currency', 'EUR'));
$currency = TransactionCurrency::where('code', $currencyPreference->data)->first();
if (is_null($currency)) {
throw new FireflyException(sprintf('No currency found with code "%s"', $currencyPreference->data));

View File

@ -0,0 +1,46 @@
<?php
/**
* ConfigurationInterface.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Support\Import\Configuration;
use FireflyIII\Models\ImportJob;
/**
* Class ConfigurationInterface
*
* @package FireflyIII\Support\Import\Configuration
*/
interface ConfigurationInterface
{
/**
* Get the data necessary to show the configuration screen.
*
* @return array
*/
public function getData(): array;
/**
* @param ImportJob $job
*
* @return ConfigurationInterface
*/
public function setJob(ImportJob $job);
/**
* Store the result.
*
* @param array $data
*
* @return bool
*/
public function storeConfiguration(array $data): bool;
}

View File

@ -0,0 +1,124 @@
<?php
/**
* CsvInitial.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Support\Import\Configuration\Csv;
use ExpandedForm;
use FireflyIII\Models\AccountType;
use FireflyIII\Models\ImportJob;
use FireflyIII\Repositories\Account\AccountRepositoryInterface;
use FireflyIII\Support\Import\Configuration\ConfigurationInterface;
use Log;
/**
* Class CsvInitial
*
* @package FireflyIII\Support\Import\Configuration
*/
class Initial implements ConfigurationInterface
{
private $job;
/**
* @return array
*/
public function getData(): array
{
/** @var AccountRepositoryInterface $accountRepository */
$accountRepository = app(AccountRepositoryInterface::class);
$accounts = $accountRepository->getAccountsByType([AccountType::DEFAULT, AccountType::ASSET]);
$delimiters = [
',' => trans('form.csv_comma'),
';' => trans('form.csv_semicolon'),
'tab' => trans('form.csv_tab'),
];
$specifics = [];
// collect specifics.
foreach (config('csv.import_specifics') as $name => $className) {
$specifics[$name] = [
'name' => $className::getName(),
'description' => $className::getDescription(),
];
}
$data = [
'accounts' => ExpandedForm::makeSelectList($accounts),
'specifix' => [],
'delimiters' => $delimiters,
'specifics' => $specifics,
];
return $data;
}
/**
* @param ImportJob $job
*
* @return ConfigurationInterface
*/
public function setJob(ImportJob $job): ConfigurationInterface
{
$this->job = $job;
return $this;
}
/**
* Store the result.
*
* @param array $data
*
* @return bool
*/
public function storeConfiguration(array $data): bool
{
/** @var AccountRepositoryInterface $repository */
$repository = app(AccountRepositoryInterface::class);
$importId = $data['csv_import_account'] ?? 0;
$account = $repository->find(intval($importId));
$hasHeaders = isset($data['has_headers']) && intval($data['has_headers']) === 1 ? true : false;
$config = $this->job->configuration;
$config['initial-config-complete'] = true;
$config['has-headers'] = $hasHeaders;
$config['date-format'] = $data['date_format'];
$config['delimiter'] = $data['csv_delimiter'];
$config['delimiter'] = $config['delimiter'] === 'tab' ? "\t" : $config['delimiter'];
Log::debug('Entered import account.', ['id' => $importId]);
if (!is_null($account->id)) {
Log::debug('Found account.', ['id' => $account->id, 'name' => $account->name]);
$config['import-account'] = $account->id;
}
if (is_null($account->id)) {
Log::error('Could not find anything for csv_import_account.', ['id' => $importId]);
}
// loop specifics.
if (isset($data['specifics']) && is_array($data['specifics'])) {
foreach ($data['specifics'] as $name => $enabled) {
// verify their content.
$className = sprintf('FireflyIII\Import\Specifics\%s', $name);
if (class_exists($className)) {
$config['specifics'][$name] = 1;
}
}
}
$this->job->configuration = $config;
$this->job->save();
return true;
}
}

View File

@ -0,0 +1,268 @@
<?php
/**
* Mapping.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Support\Import\Configuration\Csv;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Mapper\MapperInterface;
use FireflyIII\Import\MapperPreProcess\PreProcessorInterface;
use FireflyIII\Import\Specifics\SpecificInterface;
use FireflyIII\Models\ImportJob;
use FireflyIII\Support\Import\Configuration\ConfigurationInterface;
use League\Csv\Reader;
use Log;
/**
* Class Mapping
*
* @package FireflyIII\Support\Import\Configuration\Csv
*/
class Map implements ConfigurationInterface
{
/** @var array */
private $configuration = [];
/** @var array that holds each column to be mapped by the user */
private $data = [];
/** @var ImportJob */
private $job;
/** @var array */
private $validSpecifics = [];
/**
* @return array
* @throws FireflyException
*/
public function getData(): array
{
$this->configuration = $this->job->configuration;
$this->getMappableColumns();
// in order to actually map we also need all possible values from the CSV file.
$content = $this->job->uploadFileContents();
/** @var Reader $reader */
$reader = Reader::createFromString($content);
$reader->setDelimiter($this->configuration['delimiter']);
$offset = $this->configuration['has-headers'] ? 1 : 0;
$results = $reader->setOffset($offset)->fetch();
$this->validSpecifics = array_keys(config('csv.import_specifics'));
$indexes = array_keys($this->data);
$rowIndex = 0;
foreach ($results as $rowIndex => $row) {
$row = $this->runSpecifics($row);
//do something here
foreach ($indexes as $index) { // this is simply 1, 2, 3, etc.
if (!isset($row[$index])) {
// don't really know how to handle this. Just skip, for now.
continue;
}
$value = $row[$index];
if (strlen($value) > 0) {
// we can do some preprocessing here,
// which is exclusively to fix the tags:
if (!is_null($this->data[$index]['preProcessMap']) && strlen($this->data[$index]['preProcessMap']) > 0) {
/** @var PreProcessorInterface $preProcessor */
$preProcessor = app($this->data[$index]['preProcessMap']);
$result = $preProcessor->run($value);
$this->data[$index]['values'] = array_merge($this->data[$index]['values'], $result);
Log::debug($rowIndex . ':' . $index . 'Value before preprocessor', ['value' => $value]);
Log::debug($rowIndex . ':' . $index . 'Value after preprocessor', ['value-new' => $result]);
Log::debug($rowIndex . ':' . $index . 'Value after joining', ['value-complete' => $this->data[$index]['values']]);
continue;
}
$this->data[$index]['values'][] = $value;
}
}
}
foreach ($this->data as $index => $entry) {
$this->data[$index]['values'] = array_unique($this->data[$index]['values']);
}
// save number of rows, thus number of steps, in job:
$steps = $rowIndex * 5;
$extended = $this->job->extended_status;
$extended['steps'] = $steps;
$this->job->extended_status = $extended;
$this->job->save();
return $this->data;
}
/**
* @param ImportJob $job
*
* @return ConfigurationInterface
*/
public function setJob(ImportJob $job): ConfigurationInterface
{
$this->job = $job;
return $this;
}
/**
* Store the result.
*
* @param array $data
*
* @return bool
*/
public function storeConfiguration(array $data): bool
{
$config = $this->job->configuration;
foreach ($data['mapping'] as $index => $data) {
$config['column-mapping-config'][$index] = [];
foreach ($data as $value => $mapId) {
$mapId = intval($mapId);
if ($mapId !== 0) {
$config['column-mapping-config'][$index][$value] = intval($mapId);
}
}
}
// set thing to be completed.
$config['column-mapping-complete'] = true;
$this->job->configuration = $config;
$this->job->save();
return true;
}
/**
* @param string $column
*
* @return MapperInterface
*/
private function createMapper(string $column): MapperInterface
{
$mapperClass = config('csv.import_roles.' . $column . '.mapper');
$mapperName = sprintf('\\FireflyIII\\Import\Mapper\\%s', $mapperClass);
/** @var MapperInterface $mapper */
$mapper = new $mapperName;
return $mapper;
}
/**
* @return bool
*/
private function getMappableColumns(): bool
{
$config = $this->job->configuration;
/**
* @var int $index
* @var bool $mustBeMapped
*/
foreach ($config['column-do-mapping'] as $index => $mustBeMapped) {
$column = $this->validateColumnName($config['column-roles'][$index] ?? '_ignore');
$shouldMap = $this->shouldMapColumn($column, $mustBeMapped);
if ($shouldMap) {
// create configuration entry for this specific column and add column to $this->data array for later processing.
$this->data[$index] = [
'name' => $column,
'index' => $index,
'options' => $this->createMapper($column)->getMap(),
'preProcessMap' => $this->getPreProcessorName($column),
'values' => [],
];
}
}
return true;
}
/**
* @param string $column
*
* @return string
*/
private function getPreProcessorName(string $column): string
{
$name = '';
$hasPreProcess = config('csv.import_roles.' . $column . '.pre-process-map');
$preProcessClass = config('csv.import_roles.' . $column . '.pre-process-mapper');
if (!is_null($hasPreProcess) && $hasPreProcess === true && !is_null($preProcessClass)) {
$name = sprintf('\\FireflyIII\\Import\\MapperPreProcess\\%s', $preProcessClass);
}
return $name;
}
/**
* @param array $row
*
* @return array
* @throws FireflyException
*/
private function runSpecifics(array $row): array
{
// run specifics here:
// and this is the point where the specifix go to work.
foreach ($this->configuration['specifics'] as $name => $enabled) {
if (!in_array($name, $this->validSpecifics)) {
throw new FireflyException(sprintf('"%s" is not a valid class name', $name));
}
$class = config('csv.import_specifics.' . $name);
/** @var SpecificInterface $specific */
$specific = app($class);
// it returns the row, possibly modified:
$row = $specific->run($row);
}
return $row;
}
/**
* @param string $column
* @param bool $mustBeMapped
*
* @return bool
*/
private function shouldMapColumn(string $column, bool $mustBeMapped): bool
{
$canBeMapped = config('csv.import_roles.' . $column . '.mappable');
return ($canBeMapped && $mustBeMapped);
}
/**
* @param string $column
*
* @return string
* @throws FireflyException
*/
private function validateColumnName(string $column): string
{
// is valid column?
$validColumns = array_keys(config('csv.import_roles'));
if (!in_array($column, $validColumns)) {
throw new FireflyException(sprintf('"%s" is not a valid column.', $column));
}
return $column;
}
}

View File

@ -0,0 +1,263 @@
<?php
/**
* Roles.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Support\Import\Configuration\Csv;
use FireflyIII\Import\Specifics\SpecificInterface;
use FireflyIII\Models\ImportJob;
use FireflyIII\Support\Import\Configuration\ConfigurationInterface;
use League\Csv\Reader;
use Log;
/**
* Class Roles
*
* @package FireflyIII\Support\Import\Configuration\Csv
*/
class Roles implements ConfigurationInterface
{
private $data = [];
/** @var ImportJob */
private $job;
/**
* Get the data necessary to show the configuration screen.
*
* @return array
*/
public function getData(): array
{
$config = $this->job->configuration;
$content = $this->job->uploadFileContents();
// create CSV reader.
$reader = Reader::createFromString($content);
$reader->setDelimiter($config['delimiter']);
$start = $config['has-headers'] ? 1 : 0;
$end = $start + config('csv.example_rows');
// set data:
$this->data = [
'examples' => [],
'roles' => $this->getRoles(),
'total' => 0,
'headers' => $config['has-headers'] ? $reader->fetchOne(0) : [],
];
while ($start < $end) {
$row = $reader->fetchOne($start);
$row = $this->processSpecifics($row);
$count = count($row);
$this->data['total'] = $count > $this->data['total'] ? $count : $this->data['total'];
$this->processRow($row);
$start++;
}
$this->updateColumCount();
$this->makeExamplesUnique();
return $this->data;
}
/**
* Store the result.
*
* @param array $data
*
* @return bool
*/
public function storeConfiguration(array $data): bool
{
Log::debug('Now in storeConfiguration of Roles.');
$config = $this->job->configuration;
$count = $config['column-count'];
for ($i = 0; $i < $count; $i++) {
$role = $data['role'][$i] ?? '_ignore';
$mapping = isset($data['map'][$i]) && $data['map'][$i] === '1' ? true : false;
$config['column-roles'][$i] = $role;
$config['column-do-mapping'][$i] = $mapping;
Log::debug(sprintf('Column %d has been given role %s', $i, $role));
}
$this->job->configuration = $config;
$this->job->save();
$this->ignoreUnmappableColumns();
$this->setRolesComplete();
$this->isMappingNecessary();
return true;
}
/**
* @return array
*/
private function getRoles(): array
{
$roles = [];
foreach (array_keys(config('csv.import_roles')) as $role) {
$roles[$role] = trans('csv.column_' . $role);
}
return $roles;
}
/**
* @return bool
*/
private function ignoreUnmappableColumns(): bool
{
$config = $this->job->configuration;
$count = $config['column-count'];
for ($i = 0; $i < $count; $i++) {
$role = $config['column-roles'][$i] ?? '_ignore';
$mapping = $config['column-do-mapping'][$i] ?? false;
if ($role === '_ignore' && $mapping === true) {
$mapping = false;
Log::debug(sprintf('Column %d has type %s so it cannot be mapped.', $i, $role));
}
$config['column-do-mapping'][$i] = $mapping;
}
$this->job->configuration = $config;
$this->job->save();
return true;
}
/**
* @return bool
*/
private function isMappingNecessary()
{
$config = $this->job->configuration;
$count = $config['column-count'];
$toBeMapped = 0;
for ($i = 0; $i < $count; $i++) {
$mapping = $config['column-do-mapping'][$i] ?? false;
if ($mapping === true) {
$toBeMapped++;
}
}
Log::debug(sprintf('Found %d columns that need mapping.', $toBeMapped));
if ($toBeMapped === 0) {
// skip setting of map, because none need to be mapped:
$config['column-mapping-complete'] = true;
$this->job->configuration = $config;
$this->job->save();
}
return true;
}
/**
* make unique example data
*/
private function makeExamplesUnique(): bool
{
foreach ($this->data['examples'] as $index => $values) {
$this->data['examples'][$index] = array_unique($values);
}
return true;
}
/**
* @param array $row
*
* @return bool
*/
private function processRow(array $row): bool
{
foreach ($row as $index => $value) {
$value = trim($value);
if (strlen($value) > 0) {
$this->data['examples'][$index][] = $value;
}
}
return true;
}
/**
* run specifics here:
* and this is the point where the specifix go to work.
*
* @param array $row
*
* @return array
*/
private function processSpecifics(array $row): array
{
foreach ($this->job->configuration['specifics'] as $name => $enabled) {
/** @var SpecificInterface $specific */
$specific = app('FireflyIII\Import\Specifics\\' . $name);
$row = $specific->run($row);
}
return $row;
}
/**
* @return bool
*/
private function setRolesComplete(): bool
{
$config = $this->job->configuration;
$count = $config['column-count'];
$assigned = 0;
for ($i = 0; $i < $count; $i++) {
$role = $config['column-roles'][$i] ?? '_ignore';
if ($role !== '_ignore') {
$assigned++;
}
}
if ($assigned > 0) {
$config['column-roles-complete'] = true;
$this->job->configuration = $config;
$this->job->save();
}
return true;
}
/**
* @return bool
*/
private function updateColumCount(): bool
{
$config = $this->job->configuration;
$count = $this->data['total'];
$config['column-count'] = $count;
$this->job->configuration = $config;
$this->job->save();
return true;
}
/**
* @param ImportJob $job
*
* @return ConfigurationInterface
*/
public function setJob(ImportJob $job): ConfigurationInterface
{
$this->job = $job;
return $this;
}
}

View File

@ -1,242 +0,0 @@
<?php
/**
* CsvImportSupportTrait.php
* Copyright (c) 2017 thegrumpydictator@gmail.com
* This software may be modified and distributed under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License.
*
* See the LICENSE file for details.
*/
declare(strict_types=1);
namespace FireflyIII\Support\Import;
use FireflyIII\Exceptions\FireflyException;
use FireflyIII\Import\Mapper\MapperInterface;
use FireflyIII\Import\MapperPreProcess\PreProcessorInterface;
use FireflyIII\Import\Specifics\SpecificInterface;
use League\Csv\Reader;
use Log;
/**
* Trait CsvImportSupportTrait
*
* @package FireflyIII\Support\Import
*/
trait CsvImportSupportTrait
{
/**
* @return bool
*/
protected function doColumnMapping(): bool
{
$mapArray = $this->job->configuration['column-do-mapping'] ?? [];
$doMap = false;
foreach ($mapArray as $value) {
if ($value === true) {
$doMap = true;
break;
}
}
return $this->job->configuration['column-mapping-complete'] === false && $doMap;
}
/**
* @return bool
*/
protected function doColumnRoles(): bool
{
return $this->job->configuration['column-roles-complete'] === false;
}
/**
* @return array
* @throws FireflyException
*/
protected function getDataForColumnMapping(): array
{
$config = $this->job->configuration;
$data = [];
$indexes = [];
foreach ($config['column-do-mapping'] as $index => $mustBeMapped) {
if ($mustBeMapped) {
$column = $config['column-roles'][$index] ?? '_ignore';
// is valid column?
$validColumns = array_keys(config('csv.import_roles'));
if (!in_array($column, $validColumns)) {
throw new FireflyException(sprintf('"%s" is not a valid column.', $column));
}
$canBeMapped = config('csv.import_roles.' . $column . '.mappable');
$preProcessMap = config('csv.import_roles.' . $column . '.pre-process-map');
if ($canBeMapped) {
$mapperClass = config('csv.import_roles.' . $column . '.mapper');
$mapperName = sprintf('\\FireflyIII\\Import\Mapper\\%s', $mapperClass);
/** @var MapperInterface $mapper */
$mapper = new $mapperName;
$indexes[] = $index;
$data[$index] = [
'name' => $column,
'mapper' => $mapperName,
'index' => $index,
'options' => $mapper->getMap(),
'preProcessMap' => null,
'values' => [],
];
if ($preProcessMap) {
$preClass = sprintf(
'\\FireflyIII\\Import\\MapperPreProcess\\%s',
config('csv.import_roles.' . $column . '.pre-process-mapper')
);
$data[$index]['preProcessMap'] = $preClass;
}
}
}
}
// in order to actually map we also need all possible values from the CSV file.
$content = $this->job->uploadFileContents();
/** @var Reader $reader */
$reader = Reader::createFromString($content);
$reader->setDelimiter($config['delimiter']);
$results = $reader->fetch();
$validSpecifics = array_keys(config('csv.import_specifics'));
foreach ($results as $rowIndex => $row) {
// skip first row?
if ($rowIndex === 0 && $config['has-headers']) {
continue;
}
// run specifics here:
// and this is the point where the specifix go to work.
foreach ($config['specifics'] as $name => $enabled) {
if (!in_array($name, $validSpecifics)) {
throw new FireflyException(sprintf('"%s" is not a valid class name', $name));
}
$class = config('csv.import_specifics.' . $name);
/** @var SpecificInterface $specific */
$specific = app($class);
// it returns the row, possibly modified:
$row = $specific->run($row);
}
//do something here
foreach ($indexes as $index) { // this is simply 1, 2, 3, etc.
if (!isset($row[$index])) {
// don't really know how to handle this. Just skip, for now.
continue;
}
$value = $row[$index];
if (strlen($value) > 0) {
// we can do some preprocessing here,
// which is exclusively to fix the tags:
if (!is_null($data[$index]['preProcessMap'])) {
/** @var PreProcessorInterface $preProcessor */
$preProcessor = app($data[$index]['preProcessMap']);
$result = $preProcessor->run($value);
$data[$index]['values'] = array_merge($data[$index]['values'], $result);
Log::debug($rowIndex . ':' . $index . 'Value before preprocessor', ['value' => $value]);
Log::debug($rowIndex . ':' . $index . 'Value after preprocessor', ['value-new' => $result]);
Log::debug($rowIndex . ':' . $index . 'Value after joining', ['value-complete' => $data[$index]['values']]);
continue;
}
$data[$index]['values'][] = $value;
}
}
}
foreach ($data as $index => $entry) {
$data[$index]['values'] = array_unique($data[$index]['values']);
}
return $data;
}
/**
* This method collects the data that will enable a user to choose column content.
*
* @return array
*/
protected function getDataForColumnRoles(): array
{
Log::debug('Now in getDataForColumnRoles()');
$config = $this->job->configuration;
$data = [
'columns' => [],
'columnCount' => 0,
'columnHeaders' => [],
];
// show user column role configuration.
$content = $this->job->uploadFileContents();
// create CSV reader.
$reader = Reader::createFromString($content);
$reader->setDelimiter($config['delimiter']);
$start = $config['has-headers'] ? 1 : 0;
$end = $start + config('csv.example_rows');
$header = [];
if ($config['has-headers']) {
$header = $reader->fetchOne(0);
}
// collect example data in $data['columns']
Log::debug(sprintf('While %s is smaller than %d', $start, $end));
while ($start < $end) {
$row = $reader->fetchOne($start);
Log::debug(sprintf('Row %d has %d columns', $start, count($row)));
// run specifics here:
// and this is the point where the specifix go to work.
foreach ($config['specifics'] as $name => $enabled) {
/** @var SpecificInterface $specific */
$specific = app('FireflyIII\Import\Specifics\\' . $name);
Log::debug(sprintf('Will now apply specific "%s" to row %d.', $name, $start));
// it returns the row, possibly modified:
$row = $specific->run($row);
}
foreach ($row as $index => $value) {
$value = trim($value);
$data['columnHeaders'][$index] = $header[$index] ?? '';
if (strlen($value) > 0) {
$data['columns'][$index][] = $value;
}
}
$start++;
$data['columnCount'] = count($row) > $data['columnCount'] ? count($row) : $data['columnCount'];
}
// make unique example data
foreach ($data['columns'] as $index => $values) {
$data['columns'][$index] = array_unique($values);
}
$data['set_roles'] = [];
// collect possible column roles:
$data['available_roles'] = [];
foreach (array_keys(config('csv.import_roles')) as $role) {
$data['available_roles'][$role] = trans('csv.column_' . $role);
}
$config['column-count'] = $data['columnCount'];
$this->job->configuration = $config;
$this->job->save();
return $data;
}
}

View File

@ -13,6 +13,7 @@ declare(strict_types=1);
namespace FireflyIII\Support;
use \Amount as GlobalAmount;
use Carbon\Carbon;
use Crypt;
use DB;
@ -47,6 +48,11 @@ class Steam
return $cache->get(); // @codeCoverageIgnore
}
$currencyId = intval($account->getMeta('currency_id'));
// use system default currency:
if ($currencyId === 0) {
$currency = GlobalAmount::getDefaultCurrency();
$currencyId = $currency->id;
}
// first part: get all balances in own currency:
$nativeBalance = strval(
$account->transactions()
@ -264,6 +270,32 @@ class Steam
return $list;
}
/**
* @param string $amount
*
* @return string
*/
public function negative(string $amount): string
{
if (bccomp($amount, '0') === 1) {
$amount = bcmul($amount, '-1');
}
return $amount;
}
/**
* @param string $amount
*
* @return string
*/
public function opposite(string $amount): string
{
$amount = bcmul($amount, '-1');
return $amount;
}
/**
* @param $string
*
@ -299,8 +331,6 @@ class Steam
}
// parse PHP size:
/**
* @param string $amount
*

116
composer.lock generated
View File

@ -104,16 +104,16 @@
},
{
"name": "davejamesmiller/laravel-breadcrumbs",
"version": "3.0.2",
"version": "3.0.3",
"source": {
"type": "git",
"url": "https://github.com/davejamesmiller/laravel-breadcrumbs.git",
"reference": "6ca5a600003ecb52a5b5af14dad82033058604e1"
"reference": "0b0f4792dee645b0f084164aa17d4320e4bb734f"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/davejamesmiller/laravel-breadcrumbs/zipball/6ca5a600003ecb52a5b5af14dad82033058604e1",
"reference": "6ca5a600003ecb52a5b5af14dad82033058604e1",
"url": "https://api.github.com/repos/davejamesmiller/laravel-breadcrumbs/zipball/0b0f4792dee645b0f084164aa17d4320e4bb734f",
"reference": "0b0f4792dee645b0f084164aa17d4320e4bb734f",
"shasum": ""
},
"require": {
@ -124,8 +124,7 @@
"require-dev": {
"mockery/mockery": "0.9.*",
"orchestra/testbench": "3.2.*|3.3.*",
"phpunit/phpunit": "4.*",
"satooshi/php-coveralls": "0.6.*"
"phpunit/phpunit": "4.*"
},
"type": "library",
"autoload": {
@ -144,12 +143,12 @@
"homepage": "https://davejamesmiller.com/"
}
],
"description": "A simple Laravel-style way to create breadcrumbs in Laravel 4+.",
"description": "A simple Laravel-style way to create breadcrumbs in Laravel.",
"homepage": "https://laravel-breadcrumbs.readthedocs.io/",
"keywords": [
"laravel"
],
"time": "2017-01-30T21:16:53+00:00"
"time": "2017-06-24T11:10:49+00:00"
},
{
"name": "doctrine/annotations",
@ -665,16 +664,16 @@
},
{
"name": "laravel/framework",
"version": "v5.4.24",
"version": "v5.4.27",
"source": {
"type": "git",
"url": "https://github.com/laravel/framework.git",
"reference": "ec8548db26c1b147570f661128649e98f3ac0f29"
"reference": "66f5e1b37cbd66e730ea18850ded6dc0ad570404"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/laravel/framework/zipball/ec8548db26c1b147570f661128649e98f3ac0f29",
"reference": "ec8548db26c1b147570f661128649e98f3ac0f29",
"url": "https://api.github.com/repos/laravel/framework/zipball/66f5e1b37cbd66e730ea18850ded6dc0ad570404",
"reference": "66f5e1b37cbd66e730ea18850ded6dc0ad570404",
"shasum": ""
},
"require": {
@ -790,7 +789,7 @@
"framework",
"laravel"
],
"time": "2017-05-30T12:44:32+00:00"
"time": "2017-06-15T19:08:25+00:00"
},
{
"name": "laravelcollective/html",
@ -1057,16 +1056,16 @@
},
{
"name": "monolog/monolog",
"version": "1.22.1",
"version": "1.23.0",
"source": {
"type": "git",
"url": "https://github.com/Seldaek/monolog.git",
"reference": "1e044bc4b34e91743943479f1be7a1d5eb93add0"
"reference": "fd8c787753b3a2ad11bc60c063cff1358a32a3b4"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/Seldaek/monolog/zipball/1e044bc4b34e91743943479f1be7a1d5eb93add0",
"reference": "1e044bc4b34e91743943479f1be7a1d5eb93add0",
"url": "https://api.github.com/repos/Seldaek/monolog/zipball/fd8c787753b3a2ad11bc60c063cff1358a32a3b4",
"reference": "fd8c787753b3a2ad11bc60c063cff1358a32a3b4",
"shasum": ""
},
"require": {
@ -1087,7 +1086,7 @@
"phpunit/phpunit-mock-objects": "2.3.0",
"ruflin/elastica": ">=0.90 <3.0",
"sentry/sentry": "^0.13",
"swiftmailer/swiftmailer": "~5.3"
"swiftmailer/swiftmailer": "^5.3|^6.0"
},
"suggest": {
"aws/aws-sdk-php": "Allow sending log messages to AWS services like DynamoDB",
@ -1131,7 +1130,7 @@
"logging",
"psr-3"
],
"time": "2017-03-13T07:08:03+00:00"
"time": "2017-06-19T01:22:40+00:00"
},
{
"name": "mtdowling/cron-expression",
@ -1815,16 +1814,16 @@
},
{
"name": "symfony/event-dispatcher",
"version": "v2.8.21",
"version": "v2.8.22",
"source": {
"type": "git",
"url": "https://github.com/symfony/event-dispatcher.git",
"reference": "7fc8e2b4118ff316550596357325dfd92a51f531"
"reference": "1377400fd641d7d1935981546aaef780ecd5bf6d"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/event-dispatcher/zipball/7fc8e2b4118ff316550596357325dfd92a51f531",
"reference": "7fc8e2b4118ff316550596357325dfd92a51f531",
"url": "https://api.github.com/repos/symfony/event-dispatcher/zipball/1377400fd641d7d1935981546aaef780ecd5bf6d",
"reference": "1377400fd641d7d1935981546aaef780ecd5bf6d",
"shasum": ""
},
"require": {
@ -1871,7 +1870,7 @@
],
"description": "Symfony EventDispatcher Component",
"homepage": "https://symfony.com",
"time": "2017-04-26T16:56:54+00:00"
"time": "2017-06-02T07:47:27+00:00"
},
{
"name": "symfony/finder",
@ -2062,16 +2061,16 @@
},
{
"name": "symfony/polyfill-mbstring",
"version": "v1.3.0",
"version": "v1.4.0",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-mbstring.git",
"reference": "e79d363049d1c2128f133a2667e4f4190904f7f4"
"reference": "f29dca382a6485c3cbe6379f0c61230167681937"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/e79d363049d1c2128f133a2667e4f4190904f7f4",
"reference": "e79d363049d1c2128f133a2667e4f4190904f7f4",
"url": "https://api.github.com/repos/symfony/polyfill-mbstring/zipball/f29dca382a6485c3cbe6379f0c61230167681937",
"reference": "f29dca382a6485c3cbe6379f0c61230167681937",
"shasum": ""
},
"require": {
@ -2083,7 +2082,7 @@
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "1.3-dev"
"dev-master": "1.4-dev"
}
},
"autoload": {
@ -2117,20 +2116,20 @@
"portable",
"shim"
],
"time": "2016-11-14T01:06:16+00:00"
"time": "2017-06-09T14:24:12+00:00"
},
{
"name": "symfony/polyfill-php56",
"version": "v1.3.0",
"version": "v1.4.0",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-php56.git",
"reference": "1dd42b9b89556f18092f3d1ada22cb05ac85383c"
"reference": "bc0b7d6cb36b10cfabb170a3e359944a95174929"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-php56/zipball/1dd42b9b89556f18092f3d1ada22cb05ac85383c",
"reference": "1dd42b9b89556f18092f3d1ada22cb05ac85383c",
"url": "https://api.github.com/repos/symfony/polyfill-php56/zipball/bc0b7d6cb36b10cfabb170a3e359944a95174929",
"reference": "bc0b7d6cb36b10cfabb170a3e359944a95174929",
"shasum": ""
},
"require": {
@ -2140,7 +2139,7 @@
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "1.3-dev"
"dev-master": "1.4-dev"
}
},
"autoload": {
@ -2173,20 +2172,20 @@
"portable",
"shim"
],
"time": "2016-11-14T01:06:16+00:00"
"time": "2017-06-09T08:25:21+00:00"
},
{
"name": "symfony/polyfill-util",
"version": "v1.3.0",
"version": "v1.4.0",
"source": {
"type": "git",
"url": "https://github.com/symfony/polyfill-util.git",
"reference": "746bce0fca664ac0a575e465f65c6643faddf7fb"
"reference": "ebccbde4aad410f6438d86d7d261c6b4d2b9a51d"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/symfony/polyfill-util/zipball/746bce0fca664ac0a575e465f65c6643faddf7fb",
"reference": "746bce0fca664ac0a575e465f65c6643faddf7fb",
"url": "https://api.github.com/repos/symfony/polyfill-util/zipball/ebccbde4aad410f6438d86d7d261c6b4d2b9a51d",
"reference": "ebccbde4aad410f6438d86d7d261c6b4d2b9a51d",
"shasum": ""
},
"require": {
@ -2195,7 +2194,7 @@
"type": "library",
"extra": {
"branch-alias": {
"dev-master": "1.3-dev"
"dev-master": "1.4-dev"
}
},
"autoload": {
@ -2225,7 +2224,7 @@
"polyfill",
"shim"
],
"time": "2016-11-14T01:06:16+00:00"
"time": "2017-06-09T08:25:21+00:00"
},
{
"name": "symfony/process",
@ -2761,28 +2760,30 @@
},
{
"name": "barryvdh/laravel-ide-helper",
"version": "v2.3.2",
"version": "v2.4.0",
"source": {
"type": "git",
"url": "https://github.com/barryvdh/laravel-ide-helper.git",
"reference": "e82de98cef0d6597b1b686be0b5813a3a4bb53c5"
"reference": "87a02ff574f722c685e011f76692ab869ab64f6c"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/barryvdh/laravel-ide-helper/zipball/e82de98cef0d6597b1b686be0b5813a3a4bb53c5",
"reference": "e82de98cef0d6597b1b686be0b5813a3a4bb53c5",
"url": "https://api.github.com/repos/barryvdh/laravel-ide-helper/zipball/87a02ff574f722c685e011f76692ab869ab64f6c",
"reference": "87a02ff574f722c685e011f76692ab869ab64f6c",
"shasum": ""
},
"require": {
"barryvdh/reflection-docblock": "^2.0.4",
"illuminate/console": "^5.0,<5.5",
"illuminate/filesystem": "^5.0,<5.5",
"illuminate/support": "^5.0,<5.5",
"illuminate/console": "^5.0,<5.6",
"illuminate/filesystem": "^5.0,<5.6",
"illuminate/support": "^5.0,<5.6",
"php": ">=5.4.0",
"symfony/class-loader": "^2.3|^3.0"
},
"require-dev": {
"doctrine/dbal": "~2.3",
"illuminate/config": "^5.0,<5.6",
"illuminate/view": "^5.0,<5.6",
"phpunit/phpunit": "4.*",
"scrutinizer/ocular": "~1.1",
"squizlabs/php_codesniffer": "~2.3"
@ -2794,6 +2795,11 @@
"extra": {
"branch-alias": {
"dev-master": "2.3-dev"
},
"laravel": {
"providers": [
"Barryvdh\\LaravelIdeHelper\\IdeHelperServiceProvider"
]
}
},
"autoload": {
@ -2823,7 +2829,7 @@
"phpstorm",
"sublime"
],
"time": "2017-02-22T12:27:33+00:00"
"time": "2017-06-16T14:08:59+00:00"
},
{
"name": "barryvdh/reflection-docblock",
@ -3745,16 +3751,16 @@
},
{
"name": "phpunit/phpunit",
"version": "5.7.20",
"version": "5.7.21",
"source": {
"type": "git",
"url": "https://github.com/sebastianbergmann/phpunit.git",
"reference": "3cb94a5f8c07a03c8b7527ed7468a2926203f58b"
"reference": "3b91adfb64264ddec5a2dee9851f354aa66327db"
},
"dist": {
"type": "zip",
"url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/3cb94a5f8c07a03c8b7527ed7468a2926203f58b",
"reference": "3cb94a5f8c07a03c8b7527ed7468a2926203f58b",
"url": "https://api.github.com/repos/sebastianbergmann/phpunit/zipball/3b91adfb64264ddec5a2dee9851f354aa66327db",
"reference": "3b91adfb64264ddec5a2dee9851f354aa66327db",
"shasum": ""
},
"require": {
@ -3823,7 +3829,7 @@
"testing",
"xunit"
],
"time": "2017-05-22T07:42:55+00:00"
"time": "2017-06-21T08:11:54+00:00"
},
{
"name": "phpunit/phpunit-mock-objects",

View File

@ -292,6 +292,7 @@ return [
// number of example rows:
'example_rows' => 5,
'default_config' => [
'initial-config-complete' => false,
'has-headers' => false, // assume
'date-format' => 'Ymd', // assume
'delimiter' => ',', // assume

View File

@ -23,15 +23,21 @@ return [
'is_demo_site' => false,
],
'encryption' => (is_null(env('USE_ENCRYPTION')) || env('USE_ENCRYPTION') === true),
'version' => '4.5.0',
'maxUploadSize' => 5242880,
'version' => '4.6.0',
'maxUploadSize' => 15242880,
'allowedMimes' => ['image/png', 'image/jpeg', 'application/pdf'],
'list_length' => 10,
'export_formats' => [
'csv' => 'FireflyIII\Export\Exporter\CsvExporter',
],
'import_formats' => [
'csv' => 'FireflyIII\Import\Importer\CsvImporter',
'csv' => 'FireflyIII\Import\Configurator\CsvConfigurator',
],
'import_configurators' => [
'csv' => 'FireflyIII\Import\Configurator\CsvConfigurator',
],
'import_processors' => [
'csv' => 'FireflyIII\Import\FileProcessor\CsvProcessor',
],
'default_export_format' => 'csv',
'default_import_format' => 'csv',

Some files were not shown because too many files have changed in this diff Show More