Support Manager Ticket Created From Received Email With Attachment

[dait]

I have now sent an email from a client to the ticket system email account and included a short attachment to it.

 

Blesta correctly created a new ticket and inside that ticket (viewed from stuff area -> Support -> Tickets -> that ticket) I can see there is this paper clip icon with the name of the attachment, which means that Blesta correctly recognized that the message included an attachment. Below the paper clip icon and the name of the attachment the message body is correctly displayed.

 

However, if I click on the name of the attachment my browser wants to download the file, which is expected and I confirm it. But the file saved to the disk is empty (its size is 0 bytes).

Is this a bug or is this a problem of my settings somewhere?

 

Thanks!

 

 

[Michael]

have you got a uploads folder with another folder called 1 and inside that a folder called support_manager_files

 

so

 

/uploads/1/support_manager_files and ensure the uploads folder is 0777 permissions and it should upload there.

[Paul]

Mike is right, it may be a permissions issue.

[dait]

Thank you, will check.

[dait]

We have not manually created neither the folder "1", nor its subfolder "support_manager_files".

 

But "1\support_manager_files" folder does exist and I the folder's rights seem to be OK because the folder contains files and one of those files is that attachment file that I was talking about. The file has the correct size and correct contents, so the file is completely OK. I have tried to download it as a stuff member from that ticket and it still downloads zero-sized file.

 

However, despite the folder's right are OK, the file's right are probably not.

 

I will check further.

[dait]

OK, after further inspection - the files under "support_manager_files" folder are created with different rights than it is set by default. I mean if I use an account X to log into the machine create a file in that directory, it has certain rights inherited from its parent directory - i.e. "support_manager_files" folder. But if Blesta's cron creates the file it has different set of rights and does not inherit its parent folder rights correctly.

 

Please note this is Windows Server where our Blesta lives.

 

I have made an experiment:

 

I have logged into the machine and manually renamed the attachment file stored in "support_manager_files" folder. I have then created a new file with the original name of that attachment file. I have then tried to download the file from the Support Management ticket. It worked perfectly.

 

Experiment conclusion: If the file is created "normally", it inherits the rights correctly (and this work regardless the user that creates the file "normally") and is readable for all users and Blesta has no problem to work with it and provide its contents to the staff via SM ticket. However, if Blesta creates the file its rights are not inherited correctly and are not readable for all users.

 

I suspect that there is a code in Blesta that is responsible for creating these file and that it modifies the file rights in certain way. Could someone point me to that code so I could do further inspection? I assume there is chmod() call, which does the mess on Windows as could be find in many discussions over the web.

[dait]

OK, here are solutions for those who are interested. Please note that mentioned reasons are my best guess, I have not verified all my claims properly, I am just happy that it works and I want to move on:

 

1) ideal problem resolution

Blesta should not use chmod() on files on Windows, because it does not do what you expect. It breaks the access rights inheritance on Windows.

 

2) non-ideal problem resolution

Do not use CLI for CRON on Windows. It is a good security practice to run Web server under a user that can not log in on the machine, the user is just used for running the web service and is very limited. This is why if you use this security practice your CRON task should not be able to run under that user. Unfortunately, using another account for CRON task here does not work since Blesta uses chmod() function which corrupts access rights inheritance, which is described in my previous post. So, instead I suggest you to avoid running CLI for CRON and actually run e.g. wget to access your CRON page so that web server takes care of it.

 

Why is 2) non-ideal? Regardless whether you use CLI under the web server user or you avoid CLI and run it through wget, the files created by Blesta have obscure access rights. No other users will be able to access those files. If you have users for e.g. FTP that does have access to these folders, those users would not be able to access those unless they are under Administrators group. This violates ACL hierarchy that one designed on his server and thus should be considered as a bug.

 

Edit: The correct solution of this is completely different. See below.

[Cody]

OK, after further inspection - the files under "support_manager_files" folder are created with different rights than it is set by default. I mean if I use an account X to log into the machine create a file in that directory, it has certain rights inherited from its parent directory - i.e. "support_manager_files" folder. But if Blesta's cron creates the file it has different set of rights and does not inherit its parent folder rights correctly.

 

Please note this is Windows Server where our Blesta lives.

 

I have made an experiment:

 

I have logged into the machine and manually renamed the attachment file stored in "support_manager_files" folder. I have then created a new file with the original name of that attachment file. I have then tried to download the file from the Support Management ticket. It worked perfectly.

 

Experiment conclusion: If the file is created "normally", it inherits the rights correctly (and this work regardless the user that creates the file "normally") and is readable for all users and Blesta has no problem to work with it and provide its contents to the staff via SM ticket. However, if Blesta creates the file its rights are not inherited correctly and are not readable for all users.

 

I suspect that there is a code in Blesta that is responsible for creating these file and that it modifies the file rights in certain way. Could someone point me to that code so I could do further inspection? I assume there is chmod() call, which does the mess on Windows as could be find in many discussions over the web.

 

It sounds like the issue you're having here is that your CRON user does not have the same permissions as your WEBSERVER user. Ensure they have the same permissions to read/write files and you'll be fine.

Blesta does not set any special permissions on files it creates, only directories. It does this to ensure that when writing files to those directories CRON/WEBSERVER users can both use these file interchangeably. However, because Windows is funky with how it manages permissions if the user used to create the directory does not have permission to escalate the permission of that directory to allow other users to access it, it fails. Thus making the directory unwritable by other users (i.e. if CRON creates the directory then WEBSERVER can not write to it, and vice versa).

[Cody]

Why is 2) non-ideal? Regardless whether you use CLI under the web server user or you avoid CLI and run it through wget, the files created by Blesta have obscure access rights. No other users will be able to access those files. If you have users for e.g. FTP that does have access to these folders, those users would not be able to access those unless they are under Administrators group. This violates ACL hierarchy that one designed on his server and thus should be considered as a bug.

 

How your Windows users are configured is beyond the control and scope of Blesta, and thus not a bug with Blesta.

[dait]

checking

[dait]

OK, I've got it. As I mentioned previously, my previous claims were not verified, they were just guesses. I have to admit that my guess was wrong. chmod() was not the cause. Although I found a bunch of evidence that using chmod() on Windows does not do what you expect it to do and actually can harm access rights inheritance, these issues are related to using chmod() on folders only. There are no reported issues with chmod()ing files on Windows. I have verified that chmod() is not the cause by removing the only chmod() in Blesta's code and indeed, it did not help.

 

 

 

X:\blesta\up\1>icacls support_manager_files
support_manager_files SRV\WWW User Group:(I)(OI)(CI)(M)
                      SRV\Web Server User:(I)(F)
                      CREATOR OWNER:(I)(OI)(CI)(IO)(F)
                      NT AUTHORITY\SYSTEM:(I)(OI)(CI)(F)
                      BUILTIN\Administrators:(I)(OI)(CI)(F)
                      BUILTIN\Users:(I)(CI)(S,WD)
                      BUILTIN\Users:(I)(CI)(S,AD)
                      BUILTIN\Users:(I)(OI)(CI)(RX)

Successfully processed 1 files; Failed processing 0 files




X:\blesta\up\1\support_manager_files>icacls "20131006T093011+0000_b4f5b5c6fdc628a143089a0e42aee5a7.txt"
20131006T093011+0000_b4f5b5c6fdc628a143089a0e42aee5a7.txt NT AUTHORITY\SYSTEM:(F)
                                                          BUILTIN\Administrators:(F)
                                                          SRV\Cron User:(F)

Successfully processed 1 files; Failed processing 0 files




X:\blesta\up\1\support_manager_files>icacls x.txt
x.txt SRV\WWW User Group:(I)(M)
      BUILTIN\Administrators:(I)(F)
      NT AUTHORITY\SYSTEM:(I)(F)
      BUILTIN\Users:(I)(RX)

Successfully processed 1 files; Failed processing 0 files

 

What is going on here? "icacls.exe" is a program to work with ACL from command line on Windows. If you run "icacls.exe file" or "icacls.exe directory" it will just display the access rights that the file or the directory have. OK?

 

That "2013100*.txt" file is the one created by Blesta. The x.txt file is created by me manually. Both were created under support_manager_files folder. What is important here? The important thing here is the difference of their access rights.

 

You can see a bunch of inherited (I) access rights that x.txt has. Why? Because the file was created normally and no one manipulated that rights. So this file was created in a directory which does use access rights inheritance feature and so it inherited those rights. Most importantly, the BUILTIN\Users read+execute right has been inherited. Now please take a look at "2013100*.txt" file. What are its rights? The inheritance was revoked and only SYSTEM and Adminastrators have full rights + the creator user Cron User. But this means that no other user in the system is able to touch the file, read its contents or do anything with it. FTP users with full access to X:\blesta folder can not download this file. If Cron User != Web Server User (as it should not), then Web Server User can not touch the file. This is a problem, this is the bug.

 

But what is the cause? Took me a while but I found it and managed to fix it.

 

 

This actually pointed me to the right way during my analysis - see Dan Danley's comment in the discussion here - http://php.net/manual/en/function.move-uploaded-file.php

 

For those using PHP on Windows and IIS, you SHOULD set the "upload_tmp_dir" value in php.ini to some directory around where your websites directory is, create that directory, and then set the same permissions on it that you have set for your websites directory. Otherwise, when you upload a file and it goes into C:\WINDOWS\Temp, then you move it to your website directory, its permissions will NOT be set correctly. This will cause you problems if you then want to manipulate that file with something like ImageMagick's convert utility.

 

 

 

There is a known issue with PHP and files uploaded to PHP on Windows systems. In case the server has no value for upload_tmp_dir set in php.ini, uploaded files are created within the default temporary directory of the user that PHP (web server in our case) runs under. The well known problem here is that this default temporary directory is not suitable for file uploads. More correctly, the access rights of files under the default temporary directory are crippled in the way we can see in case of that "2013100*.txt" file. In other words, files created in default TEMP on Windows do not inherit access rights. This is not a bug or mistake in default Windows behavior, this is an intention and it should not be manipulated, it should be kept the way it is. Files created in default TEMP are not intended to be moved to other folders. But this is what is commonly done with uploaded files. This is exactly why PHP on Windows should have upload_tmp_dir defined and set to a directory that is not within the reach of your web presentation and also is not the default TEMP directory. You should create a new folder for this. If you configure upload_tmp_dir properly, your uploaded files will be created there and the file rights will be just OK even if you move the file to another directory, which is what you commonly do.

 

OK, so where is the problem now? The problem is that we do have upload_tmp_dir set in php.ini properly. It is just Blesta that ignores it.

 

Here is the responsible function from plugins\support_manager\components\email_parser\email_parser.php:

 

	public function getAttachments(MimeMailParser $email) {
		
		$files = array();

		$tmp_dir = "/tmp";
		if (function_exists("sys_get_temp_dir"))
			$tmp_dir = sys_get_temp_dir();
		
		foreach ($email->getAttachments() as $attachment) {
...

 

 

SOLUTION: I have rewritten this code to this:

 

 

	public function getAttachments(MimeMailParser $email) {
		
		$files = array();

		$tmp_dir = "/tmp";
		$sgt_dir = function_exists("sys_get_temp_dir") ? sys_get_temp_dir() : $tmp_dir;
		$upl_dir = function_exists("ini_get") ? ini_get("upload_tmp_dir") : $sgt_dir;
		$tmp_dir = $upl_dir ? $upl_dir : $sgt_dir;

		foreach ($email->getAttachments() as $attachment) {

 

In other words, the original Blesta code goes to temporary directory. My code goes to upload_tmp_dir, which is the folder that should be used for uploads and similar operations.

 

 

Blesta in its components\upload\upload.php has this code:

 

			if ($this->uploaded_files)
				$result = move_uploaded_file($index !== null ? $file['tmp_name'][$index] : $file['tmp_name'], $this->upload_path . $new_file_name);
			else
				$result = rename($index !== null ? $file['tmp_name'][$index] : $file['tmp_name'], $this->upload_path . $new_file_name);

In case of file being downloaded from IMAP, it is the rename() function that is called to move the file from the temporary directory to "1\support_manager_files" folder. If there was copy + delete, it would work. rename() just moves (renames) existing file and its rights are intact, hence if they are wrong (because the file was created in the default temporary directory) then will be wrong after renaming too.

 

So, this has nothing to do with Cron and Web users, nothing to do with chmod(), this is about using the default temporary directory for the purpose for which it should not be used.

 

Finally, we have the result of a new attachment that was processed after the SOLUTION was applied to our Blesta 3.0.4:

 

 

X:\blesta\up\1\support_manager_files>icacls "20131008T095510+0000_94e4df435b1b2d6e0b34ba10faf0a0c2._txt"
20131008T095510+0000_94e4df435b1b2d6e0b34ba10faf0a0c2._txt SRV\Web Server User:(I)(M)
                                                           SRV\WWW User Group:(I)(RX)
                                                           SRV\Web Server User:(I)(F)
                                                           NT AUTHORITY\SYSTEM:(I)(F)
                                                           BUILTIN\Administrators:(I)(F)
                                                           BUILTIN\Users:(I)(RX)

Successfully processed 1 files; Failed processing 0 files

 

Beautiful, right? Perfectly correct access rights.

 

I hope that this evidence is enough for you to admit this is a bug in Blesta. Feel free to use my patch, free of charge, free to use license for whatever purpose forever. Windows rights are not funky, they just work on different principles than UNIX/Linux. I admit that understanding of Windows rights and principles might be crucial for understanding of what I am talking about here, but PLEASE at least try if you are not familiar with Windows.

 

Also, please note that you use sys_get_temp_dir (and e.g. store it to temp_dir value, which is used on many many places) function on other places in Blesta's code. I highly suggest that you check all places in the code in which this default temporary directory from sys_get_temp_dir is used. If in any case you create a file there and then move it (rename) somewhere else then this is likely to cause similar problem again.

[Cody]

I've created CORE-808 to fix the issue with EmailParser::getAttachments() incorrectly using the PHP function sys_get_temp_dir() to determine the temp directory. Instead it should be using the temp directory setting as defined in [settings] > [system] > [General] > [basic Setup].

[dait]

Thank you for accepting. However, [settings] > [system] > [General] > [basic Setup] does only allows you to set up temp directory and upload directory and neither should be used for this purpose. What we are talking about here is temp upload directory as I described in my post. If you change it to Temp Directory from [basic Setup] the bug will still be there because for most actions the default temporary directory is just good to be used (for things like create a temp file, do something with it and delete it). It is just unsuitable for actions where you want to move the file somewhere else.

 

So I suggest to reconsider your fix as it would not fix anything. It should be OK to left [basic Setup] Temp Directory with the default value which is the default temporary directory.

 

You have this hint - if your solution does not work with the default values, it is probably not good. And you will put all your Windows users to troubles if you do it that way.

 

There is really no why not to use what PHP offers you here - upload_tmp_dir. If you want to allow users to have it changeable, OK, create a new setting under [basic Setup], but please do not mix it with Temp Directory. That is something else.

[Cody]

Thank you for accepting. However, [settings] > [system] > [General] > [basic Setup] does only allows you to set up temp directory and upload directory and neither should be used for this purpose. What we are talking about here is temp upload directory as I described in my post. If you change it to Temp Directory from [basic Setup] the bug will still be there because for most actions the default temporary directory is just good to be used (for things like create a temp file, do something with it and delete it). It is just unsuitable for actions where you want to move the file somewhere else.

 

Why is that? Can't you simply set the proper permissions to your temp directory, or just update [settings] > [system] > [General] > [basic Setup] to point to the directory you setup in your php.ini file?

 

So I suggest to reconsider your fix as it would not fix anything. It should be OK to left [basic Setup] Temp Directory with the default value which is the default temporary directory.

 

You have this hint - if your solution does not work with the default values, it is probably not good. And you will put all your Windows users to troubles if you do it that way.

 

The purpose of the temp directory is to write files that may be moved, read, or written to. Again, we come back to a permission issue. It isn't satisfactory to say the temp directory as defined in the settings is only good for writing/reading from but not for moving files out of. That's just silly.

[dait]

Please Google something about upload_tmp_dir, why does it exist and what are its problems on Windows.

 

Just think about why upload_tmp_dir even exists? Why someone introduced it to PHP? It is there and you just need to use it, why you refuse to do that? There is no reason not to use upload_tmp_dir. Why is it called upload_tmp_dir? Why not just tmp_dir if it was the same?!

 

But if you think the default rights of Windows default temporary directory should be manipulated then the whole discussion is pointless. I can't see why you keep persisting on your UNIX/Linux based point of view when we are talking about  Windows!?

[Cody]

Please Google something about upload_tmp_dir, why does it exist and what are its problems on Windows.

 

Just think about why upload_tmp_dir even exists? Why someone introduced it to PHP? It is there and you just need to use it, why you refuse to do that? There is no reason not to use upload_tmp_dir. Why is it called upload_tmp_dir? Why not just tmp_dir if it was the same?!

 

Why are you trying to be condescending? Is it because you don't understand the problem? Because that's what it looks like.

 

The temp directory in Blesta settings is for reading, writing, and moving files. That's its sole purpose. This value is configurable and must be writable by Blesta. The entire system should use this setting whenever a temp directory is needed. That is why I created CORE-808, because the support manager does not use that setting when dealing with email attachments AND IT SHOULD because where Blesta writes files to should be predictable.

 

But if you think the default rights of Windows default temporary directory should be manipulated then the whole discussion is pointless. I can't see why you keep persisting on your UNIX/Linux based point of view when we are talking about  Windows!?

 

Arguing the default permission on Windows is a moot point. Look, I run Blesta on Windows too. I don't have the problems you're having because my permissions just work. Maybe that's because I'm using WAMP. Regardless, the issue you're experiencing is not isolated to Windows. It's a permission issue. It can happen to users on Linux just as easily as it can happen to users on Windows.

[dait]

I have analyzed the issue for you, describe the problem to its very detail so you could just take the code and fix the bug. I would expect a little thank you, but instead you just twist the whole issue into something else. How should I be happy about it?

 

You have not explained this: If a new user installs Blesta on Windows, he would have the default temporary directory there as Basic Settings Temp Directory. And it would not work. This is a bug on itself. Your solution does nothing about it. What you are doing here is creating glitches that your users will fall into. If you think this is a right way to go then please just confirm and we are done here. There is no point to discuss anything further. In such a case you do not care about your users, you do not want to create a glitch-free software that would just work. You would be creating something that would be hard to setup on production machine.

 

I believe you that you have a testing Windows machine with WAMP. I do not believe that you run Blesta on Windows in production environment.

 

I do not say this to offend you. I just know that your solution to this problem is not a real solution. You are probably a good Linux developer and you know something little about Windows and you are capable to work with Windows on a certain level. But from your comments it seems that your experience is not on the level it is required here. And I offered you a help with this. It is up to you how whether you want to accept this offer or not. If not, just tell me, OK? I will back off and ask you for a refund for my Blesta license and will move to another provider.

[Paul]

Hi dait,

 

We understand that the default setting for the temp directory on a Windows install may not be correct, and if not it should be resolved.

 

The number of Windows users is very small compared to Linux users, so it's more likely some things will need to be tweaked to work better for Windows and we're open to that and we appreciate your suggestions.

 

I guess what I don't understand is whether or not you're able to work around this issue for now. It looks like you came up with a solution above. Will that work for you for now?

 

Ultimately, I think it would be helpful to have more discussion about this. Feedback from other Windows users would be great.

 

Thanks

[dait]

Hi Paul, you could see in my history of posts here that the number of issues I am reporting is quite big. I am just trying to turn Blesta on our machine to a usable piece of software. The reason why I chose Blesta was the open source code that I must say is written in a good code style and I find it easy to read.

 

I am just unhappy that the current quality is on the level of a freeware solution and that I paid a bunch of bucks for it. But I can live with that if we can cooperate together and move Blesta on another level. For this, however, I really need positive thinking, fast reactions and a little trust. I do not need "this is not a bug" under half of my posts. I need you to agree that if something does not work in its defaults, it is a bug. There are very few exceptions to this and all such configurations have to be part of the installation process.

 

I can not handle three things:

 

1) I do not want to create and manage my own version of Blesta that would be better than the official version just because we do not agree on what is and what is not a bug.

 

2) I do not want to spend whole days debugging Blesta on things that I reported days ago and just no one replied.

 

3) Wait several days for support tickets to be answered. I am not sure whether I should write things here on there, but either way, I do not want to way. I need something in production in less than 2 months.

 

 

If we can not agree on these crucial things then OK, please refund my license and good luck with your software.

[Paul]

Hi Paul, you could see in my history of posts here that the number of issues I am reporting is quite big. I am just trying to turn Blesta on our machine to a usable piece of software. The reason why I chose Blesta was the open source code that I must say is written in a good code style and I find it easy to read.

 

I am just unhappy that the current quality is on the level of a freeware solution and that I paid a bunch of bucks for it. But I can live with that if we can cooperate together and move Blesta on another level. For this, however, I really need positive thinking, fast reactions and a little trust. I do not need "this is not a bug" under half of my posts. I need you to agree that if something does not work in its defaults, it is a bug. There are very few exceptions to this and all such configurations have to be part of the installation process.

 

Please understand that all bugs must be confirmed. As we do not currently have an IIS environment, it's difficult for us to do so when they are Windows only related. We also rely, at least partly, on the community to confirm bugs. Different users may experience different things, and a bug may be multi-faceted, requiring a more broad fix.

 

I can not handle three things:

 

1) I do not want to create and manage my own version of Blesta that would be better than the official version just because we do not agree on what is and what is not a bug.

 

You certainly shouldn't have to maintain your own build, and we don't recommend it.

 

2) I do not want to spend whole days debugging Blesta on things that I reported days ago and just no one replied.

 

Not all bugs reports are replied to immediately.

 

3) Wait several days for support tickets to be answered. I am not sure whether I should write things here on there, but either way, I do not want to way. I need something in production in less than 2 months.

 

Bug reports and feature requests are best for the forum. When a ticket is submitted, it's typically reviewed by one person, or escalated to another person. On the forums, it's open to everyone.

 

I'll take a look at some of your other threads.

[dait]

To summarize this up - I am OK with what you wrote. I will give it a shot once more. I will do my best to overcome all of the obstacles, but I hope that in case I find this too difficult (for whatever reason - too many issues, lack of support, different points of view on security/bugs related definitions such as not accepting "if something does not work with defaults it is a bug", whatever) for me that you will not refuse to issue a refund to me.

 

There are quite many opened issues where I now wait for your answers and how do you handle them will tell me whether I can go with Blesta in long term or not. So, let's work on it.

[dait]

Paul: Could you please tell me how this issue was handled?