Issues with creating new projects

11 posts / 0 new
Last post
gin
Issues with creating new projects

1) Workflows

I have a set of site-wide workflows which should be used for all projects. However, when creating a new project from a template, tasks cannot be created unless I first save the workflow. I get this error:

Process item [process_name] has not been created. Please save your pipeline in the Project Workflow Editor to refresh the processes.

Then I have to do the following in order to get things how I expect them to be:

  1. Go to Create Workflow page, select the global workflow and save
  2. project_code for global workflow is now set to P1_Project_Code (or whatever)
  3. change project_code of global workflow back to blank

Why is it necessary for me to go through all these extra steps? Did I do something wrong in setup and is there someway for me to fix this?

2) Group security

Link & Gear Menu security for groups don't seem to inherit the way I expect it to.

I have a group with access level medium, then I added link and gear menu security for the template project. However, these rules don't carry over when creating a new project. Is there a way to make it? I've even tried adding something like "<rule group="link" element="*" project="*" access="allow"/>", but it does not apply to new projects, and I still have to go in and tick all the boxes again.

remko

For 1:  the pipeline table existed a long time ago however it was decided that too much information was required to adequately describe the definition of the pipeline.  Site-wide pipelines were falling a little out of favor because of the number of dependencies that were project specific (such as triggers).  More and more, the approach taken was to use project specific pipelines and then copy them for each project and also an spt_process table was created to store more information about each process.  Fast forward now and it seems that spt_process entries need to exist, but the templating process does not copy them.  Going to the pipeline editor and pressing save automatically generates the spt_process entries and effectively removes the error.

So, I don't think you're doing anything wrong ... it's just that the templating script is woefully out of date.  This is likely because plugins are a much more powerful way of capturing elements of a project and introducing them into a new project.   However, admittedly, there is no plugin manifest that captures the entire project like the templating does.

 

remko

For 2: it may be that the groups are project specific.  Does the entry for the groups have a project_code filled out?

gin

Hi remko,

1) Regarding the spt_process table, you're referring to the one in the newly created project db? Because I did a comparison before and after I saved the pipeline and it's identical. Moreover, of the entries in the spt_process table, none of them have the same pipeline code as the global or the project pipelines.

2) The group has no project_code.

remko

There error occurs in src/pyasm/command/workflow.py

        search = Search("config/process")
        search.add_filter("pipeline_code", pipeline.get_code())
        search.add_filter("process", process)
        process_sobj = search.get_sobject()

        #print "callback process: ", process, pipeline.get_code()
        if not process_sobj:
            raise TacticException('Process item [%s] has not been created. Please save your pipeline in the Project Workflow Editor to refresh the processes.'%process)
 

This means that the entry for spt_process cannot be found.

So is this correct.  When you create a new project, there are entries in the spt_process table in the db of the newly create project? And they do not have the correct pipeline code?  Or are there no entries?  Either way would result in the error as shown in the code above.

gin

Hi remko, I accidentally gave you incorrect information early. It appears that the spt_process table IS updating after saving the workflow. I was using phpPgAdmin to monitor the tables and it has some wierd refresh issue, so I didn't notice the changes.

So I've confirmed that replacing the spt_process table accurately does get rid of that error. Which brings me to another wierd issue.

I've replaced the spt_process table for Project_Template (45 rows, pipeline_code: "global/*"). However, newly created Project_A has a spt_process table of 135 rows, none of which have the correct pipeline_code (they look like they came from several old projects). Where are they all coming from? I've currently only got the one template project, and the sthpw spt_process is empty.

Incidentally, on creating Project_B the spt_process table has 180 rows, the additional 45 rows have the pipeline code project_template/*. I didn't bother to rename and reclaim the pipelines for Project_Template, so Project_C has the same spt_process table as Project_B. Still, where is the data coming from? The entry for project_template/__TEMPLATE__ is blank.

remko

Honestly, we haven't used templates in a long time.  How are you generating templates:  Is it:

src/bin/util/create_template.py

If so, I did go and look back at the code that does the templating in:

src/tactic/command/project_template_cmd.py

It lists the exported search types as:

        search_types = [
            "config/custom_script",
            "config/widget_config",
            "config/naming",
            "config/client_trigger",
            "config/process",
            "config/trigger",
            "config/url",
        ]
 

Note that "config/process" is suspicously missing.  It should probably be in that list.

gin

Ah, no. What I did was take the VFX template, make changes, then save that as Project_Template. Then projects were created from that.

Are there any docs which are more up-to-date? Because from reading the existing docs, I got the impression that templates were the way to go.

remko

The update button does the same as the command line I wrote.  It does use TACTIC plugins in background but packaged in a way to capture an entire project configuration definition.  I do think this is the correct way to do this (a long time ago we had another templating script and I was confusing the two).  It is possible that plugin manifest or definition is out of date.

remko

The actual plugin generate would be located in <tactic_data_dir>/templates/<project_code>.  This is the definition of the project. 

The sthpw/pipeline table is global for all projects where as spt_process is project specific.  I think there was some idea that you could have a site-wide pipeline but make project specific customizations on the node processes, but it's definitely an old wart in the design.

Nevertheless, inside is a manifest file as well as a bunch of spt files.  The file "config_process.spt" has all of the entries that it will import into the new project.  These will have a pipeline code column and they should match the code of the pipeline in the "sthpw_pipeline.spt".  One thing to be careful is to actually name your codes explicitly rather than relying on TACTIC to name it for you.  This reduces the chance of conflict with another pipeline created somewhere else. 

Since you processes are always going up in increments of 45 and there is some code dispartiy, saving the pipeline creates another set up processes in the pipeline and updating the template saves the new ones and the old ones.

Take a look at those spt files and maybe there is some clue as to what is wrong.

gin

Hi remko, you're right, the "config_process.spt" file did have all those entries. Since I don't have the time or energy to fix the root problem, I just rewrote the "config_process.spt" to have the data I wanted. After that, newly created projects do have correct spt_process entries.

However.

I am still getting this error when trying to create tasks: "Process item [process_name] has not been created. Please save your pipeline in the Project Workflow Editor to refresh the processes."

What does fix this (and I don't know whyyyy ;_;) is replacing the data of the spt_process table by importing with a CSV with identical data. Specifically, I SSH'd into the server, logged into psql and "\COPY spt_process FROM '/path/spt_process.csv' WITH (FORMAT CSV, HEADER);". Then I could create tasks with no problems :P

Below is the first entry of both files. The relevant data is identical as far as I can tell. I thought at first it might be the timestamp, since my solution Nulls out the timestamp, but no joy.

config_process.spt

#-- Start Entry --#
insert = SearchType.create('config/process')
insert.set_value('code', """SPT_PROCESS00001""")
insert.set_value('process', """3D_fur""")
insert.set_value('pipeline_code', """global/asset""")
insert.set_value('subcontext_options', """(main)""")
insert.set_value('sort_order', 1)
#-- End Entry --#

spt_process.csv

id,code,pipeline_code,process,color,sort_order,timestamp,s_status,checkin_mode,subcontext_options,checkin_validate_script_path,checkin_options_view,sandbox_create_script_path,context_options,description,repo_type,transfer_mode,workflow,subpipeline_code
1,SPT_PROCESS00001,global/asset,3D_fur,,1,,,,(main),,,,,,,,,